[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:44:11 dist-test-slave-vkjp krb5kdc[25448](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:44:11 dist-test-slave-vkjp krb5kdc[25448](info): set up 2 sockets
Aug 10 14:44:11 dist-test-slave-vkjp krb5kdc[25448](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260810 14:44:12.913733 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.226s user 0.002s sys 0.015s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:44:13 dist-test-slave-vkjp krb5kdc[25448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:44:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:44:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:44:13.100708 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:40989
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:40989
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:13.443549 25464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:13.443888 25464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:13.444075 25464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:13.451493 25464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:13.451614 25464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:13.451757 25464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:13.451872 25464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:13.451960 25464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:44:13.462473 25464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:40989
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:40989
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:13.463943 25464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:13.465340 25464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:13.478641 25470 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:13.478583 25471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:13.478678 25473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:13.478971 25464 server_base.cc:1034] running on GCE node
I20260810 14:44:13.480028 25464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:13.481539 25464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:13.482807 25464 hybrid_clock.cc:648] HybridClock initialized: now 1786373053482722 us; error 82 us; skew 500 ppm
I20260810 14:44:13.483268 25464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:13 dist-test-slave-vkjp krb5kdc[25448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373053, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:13.487126 25464 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:44:13.488727 25464 webserver.cc:460] Webserver started at http://127.24.215.190:35519/ using document root <none> and password file <none>
I20260810 14:44:13.489435 25464 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:13.489603 25464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:13.489928 25464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:13.493263 25464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "78036313e8514b7db8dc8a45830ba085"
format_stamp: "Formatted at 2026-08-10 14:44:13 on dist-test-slave-vkjp"
server_key: "93aa8639b5125649cd9a0ea17dfed976"
server_key_iv: "2555608f5177098b1240f5691b5231c4"
server_key_version: "encryptionkey@0"
I20260810 14:44:13.494210 25464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "78036313e8514b7db8dc8a45830ba085"
format_stamp: "Formatted at 2026-08-10 14:44:13 on dist-test-slave-vkjp"
server_key: "93aa8639b5125649cd9a0ea17dfed976"
server_key_iv: "2555608f5177098b1240f5691b5231c4"
server_key_version: "encryptionkey@0"
I20260810 14:44:13.499997 25464 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 14:44:13.503733 25482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:13.505158 25464 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:44:13.505363 25464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "78036313e8514b7db8dc8a45830ba085"
format_stamp: "Formatted at 2026-08-10 14:44:13 on dist-test-slave-vkjp"
server_key: "93aa8639b5125649cd9a0ea17dfed976"
server_key_iv: "2555608f5177098b1240f5691b5231c4"
server_key_version: "encryptionkey@0"
I20260810 14:44:13.505604 25464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:13.539266 25464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:13.543896 25464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:13.544214 25464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:13.564302 25464 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:40989
I20260810 14:44:13.564309 25550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:40989 every 8 connection(s)
I20260810 14:44:13.565778 25464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:44:13.568418 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 25464
I20260810 14:44:13.568742 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:44:13.569792 25551 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:13.570736 25438 external_mini_cluster.cc:1428] Setting key b980ac139f387c63e7b0248b57d4f35c
I20260810 14:44:13.581584 25551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085: Bootstrap starting.
I20260810 14:44:13.584739 25551 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:13.586006 25551 log.cc:824] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:13.589949 25551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085: No bootstrap required, opened a new log
I20260810 14:44:13.599531 25551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78036313e8514b7db8dc8a45830ba085" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40989 } }
I20260810 14:44:13.599947 25551 raft_consensus.cc:374] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:13.600070 25551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78036313e8514b7db8dc8a45830ba085, State: Initialized, Role: FOLLOWER
I20260810 14:44:13.600745 25551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78036313e8514b7db8dc8a45830ba085" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40989 } }
I20260810 14:44:13.600994 25551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:44:13.601167 25551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:44:13.601361 25551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [term 0 FOLLOWER]: Advancing to term 1
Aug 10 14:44:13 dist-test-slave-vkjp krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373053, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:13.603885 25551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78036313e8514b7db8dc8a45830ba085" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40989 } }
I20260810 14:44:13.604488 25551 leader_election.cc:302] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 78036313e8514b7db8dc8a45830ba085; no voters:
I20260810 14:44:13.605299 25551 leader_election.cc:288] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:44:13.605509 25556 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:13.607577 25551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:44:13.612396 25556 raft_consensus.cc:686] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [term 1 LEADER]: Becoming Leader. State: Replica: 78036313e8514b7db8dc8a45830ba085, State: Running, Role: LEADER
I20260810 14:44:13.612897 25556 consensus_queue.cc:237] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78036313e8514b7db8dc8a45830ba085" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40989 } }
I20260810 14:44:13.615252 25553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:13.574631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55374 (local address 127.24.215.190:40989)
0810 14:44:13.575578 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:13.575599 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:13.576788 (+ 1189us) server_negotiation.cc:407] Connection header received
0810 14:44:13.586074 (+ 9286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:13.586098 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:13.586598 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:13.587058 (+ 460us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:13.591556 (+ 4498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:13.593805 (+ 2249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:13.595741 (+ 1936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:13.596497 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:13.596731 (+ 234us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:13.602933 (+ 6202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:13.602943 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:13.602957 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:13.602998 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:13.607756 (+ 4758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:13.608946 (+ 1190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:13.608957 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:13.608969 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:13.609235 (+ 266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:13.609976 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:13.609985 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:13.609988 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:13.611170 (+ 1182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:13.611653 (+ 483us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:13.613240 (+ 1587us) server_negotiation.cc:299] Negotiation successful
0810 14:44:13.613656 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":467,"threads_started":1}
I20260810 14:44:13.616371 25557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "78036313e8514b7db8dc8a45830ba085" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78036313e8514b7db8dc8a45830ba085" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40989 } } }
I20260810 14:44:13.616722 25557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:13.617249 25558 sys_catalog.cc:455] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 78036313e8514b7db8dc8a45830ba085. Latest consensus state: current_term: 1 leader_uuid: "78036313e8514b7db8dc8a45830ba085" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "78036313e8514b7db8dc8a45830ba085" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40989 } } }
I20260810 14:44:13.617547 25558 sys_catalog.cc:458] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085 [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:13.624651 25563 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:44:13.630403 25563 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:44:13.638573 25563 catalog_manager.cc:1347] Generated new cluster ID: 4dc7ce047ecf42cda0c45664f363b21e
I20260810 14:44:13.638687 25563 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:44:13.652701 25563 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:44:13.654672 25563 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:44:13.663760 25563 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 78036313e8514b7db8dc8a45830ba085: Generated new TSK 0
I20260810 14:44:13.664592 25563 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:44:13.711474 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:40989
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:14.062894 25579 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:14.063298 25579 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:14.063485 25579 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:14.071447 25579 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:14.071604 25579 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:14.071789 25579 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:44:14.082072 25579 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:40989
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:14.083515 25579 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:14.085033 25579 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:14.098850 25585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:14.098883 25586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:14.098801 25588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:14.099833 25579 server_base.cc:1034] running on GCE node
I20260810 14:44:14.100509 25579 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:14.101497 25579 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:14.102734 25579 hybrid_clock.cc:648] HybridClock initialized: now 1786373054102679 us; error 51 us; skew 500 ppm
I20260810 14:44:14.103068 25579 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:14 dist-test-slave-vkjp krb5kdc[25448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373054, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:14.106294 25579 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:44:14.107935 25579 webserver.cc:460] Webserver started at http://127.24.215.129:35161/ using document root <none> and password file <none>
I20260810 14:44:14.108625 25579 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:14.108772 25579 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:14.109163 25579 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:14.112310 25579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "81f08b0b88494e569c711dfc673b4afd"
format_stamp: "Formatted at 2026-08-10 14:44:14 on dist-test-slave-vkjp"
server_key: "35efbb28d0f4c96eb25b9f508f2d0ab1"
server_key_iv: "1dad24c718fd64157d074d52445a0cb8"
server_key_version: "encryptionkey@0"
I20260810 14:44:14.113090 25579 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "81f08b0b88494e569c711dfc673b4afd"
format_stamp: "Formatted at 2026-08-10 14:44:14 on dist-test-slave-vkjp"
server_key: "35efbb28d0f4c96eb25b9f508f2d0ab1"
server_key_iv: "1dad24c718fd64157d074d52445a0cb8"
server_key_version: "encryptionkey@0"
I20260810 14:44:14.118391 25579 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:44:14.122016 25596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:14.123396 25579 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:44:14.123579 25579 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "81f08b0b88494e569c711dfc673b4afd"
format_stamp: "Formatted at 2026-08-10 14:44:14 on dist-test-slave-vkjp"
server_key: "35efbb28d0f4c96eb25b9f508f2d0ab1"
server_key_iv: "1dad24c718fd64157d074d52445a0cb8"
server_key_version: "encryptionkey@0"
I20260810 14:44:14.123842 25579 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:14.140496 25579 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:14.144936 25579 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:14.145239 25579 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:14.146219 25579 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:14.147601 25579 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:14.147707 25579 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:14.147857 25579 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:14.147930 25579 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:14.184978 25579 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:42609
I20260810 14:44:14.185029 25728 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:42609 every 8 connection(s)
I20260810 14:44:14.186313 25579 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:44:14.191521 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 25579
I20260810 14:44:14.191762 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:44:14.192440 25438 external_mini_cluster.cc:1428] Setting key 1fc59102fadee3449871b57aa507209b
Aug 10 14:44:14 dist-test-slave-vkjp krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373054, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:14.208391 25731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:14.189153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:53233 (local address 127.24.215.190:40989)
0810 14:44:14.189718 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:14.189724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:14.190727 (+ 1003us) server_negotiation.cc:407] Connection header received
0810 14:44:14.192953 (+ 2226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:14.192959 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:14.193070 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:14.193205 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:14.195179 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:14.195999 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:14.197666 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:14.198106 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:14.198270 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:14.202353 (+ 4083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:14.202359 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:14.202363 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:14.202397 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:14.205105 (+ 2708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:14.205870 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:14.205875 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:14.205877 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:14.205999 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:14.206463 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:14.206467 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:14.206469 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:14.206730 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:14.206902 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:14.207767 (+ 865us) server_negotiation.cc:299] Negotiation successful
0810 14:44:14.208013 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":328,"threads_started":1}
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
I20260810 14:44:14.209566 25732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:14.189767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:40989 (local address 127.24.215.129:53233)
0810 14:44:14.190544 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:14.190592 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:14.192658 (+ 2066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:14.193506 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:14.193520 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:14.194264 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:14.194963 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:14.194984 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:14.196195 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:14.196199 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:14.197503 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:14.197518 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:14.198379 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:14.198389 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:14.198658 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:14.199463 (+ 805us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:14.199488 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:14.202116 (+ 2628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:14.205256 (+ 3140us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:14.205266 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:14.205283 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:14.205722 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:14.206116 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:14.206121 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:14.206125 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:14.206332 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:14.206874 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:14.206882 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:14.207319 (+ 437us) client_negotiation.cc:769] Sending connection context
0810 14:44:14.207647 (+ 328us) client_negotiation.cc:240] Negotiation successful
0810 14:44:14.208214 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":279,"threads_started":1}
I20260810 14:44:14.211457 25729 heartbeater.cc:347] Connected to a master server at 127.24.215.190:40989
I20260810 14:44:14.211843 25729 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
I20260810 14:44:14.212666 25729 heartbeater.cc:511] Master 127.24.215.190:40989 requested a full tablet report, sending...
I20260810 14:44:14.215152 25504 ts_manager.cc:194] Registered new tserver with Master: 81f08b0b88494e569c711dfc673b4afd (127.24.215.129:42609)
I20260810 14:44:14.217383 25504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:53233
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:44:14.238008 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:40989
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:14.600613 25735 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:14.600894 25735 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:14.601068 25735 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:14.608755 25735 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:14.608871 25735 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:14.609053 25735 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:44:14.619189 25735 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:40989
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:14.620600 25735 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:14.621989 25735 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:14.636219 25741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:14.636196 25745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:14.636245 25742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:14.637189 25735 server_base.cc:1034] running on GCE node
I20260810 14:44:14.637813 25735 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:14.638783 25735 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:14.640012 25735 hybrid_clock.cc:648] HybridClock initialized: now 1786373054639939 us; error 65 us; skew 500 ppm
I20260810 14:44:14.640347 25735 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:14 dist-test-slave-vkjp krb5kdc[25448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373054, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:14.643631 25735 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:44:14.645229 25735 webserver.cc:460] Webserver started at http://127.24.215.130:42173/ using document root <none> and password file <none>
I20260810 14:44:14.645895 25735 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:14.646000 25735 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:14.646267 25735 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:14.649210 25735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "14d8c2bba3804b78b591007daa217776"
format_stamp: "Formatted at 2026-08-10 14:44:14 on dist-test-slave-vkjp"
server_key: "2c6194827a889b900f03283526231d37"
server_key_iv: "db379696938c6db09d9c9e0cb85ddfaa"
server_key_version: "encryptionkey@0"
I20260810 14:44:14.649941 25735 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "14d8c2bba3804b78b591007daa217776"
format_stamp: "Formatted at 2026-08-10 14:44:14 on dist-test-slave-vkjp"
server_key: "2c6194827a889b900f03283526231d37"
server_key_iv: "db379696938c6db09d9c9e0cb85ddfaa"
server_key_version: "encryptionkey@0"
I20260810 14:44:14.655308 25735 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 14:44:14.658993 25754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:14.660470 25735 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:44:14.660640 25735 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "14d8c2bba3804b78b591007daa217776"
format_stamp: "Formatted at 2026-08-10 14:44:14 on dist-test-slave-vkjp"
server_key: "2c6194827a889b900f03283526231d37"
server_key_iv: "db379696938c6db09d9c9e0cb85ddfaa"
server_key_version: "encryptionkey@0"
I20260810 14:44:14.660833 25735 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:14.674857 25735 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:14.679366 25735 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:14.679682 25735 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:14.680698 25735 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:14.682080 25735 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:14.682155 25735 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:14.682303 25735 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:14.682382 25735 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:14.719319 25735 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:37595
I20260810 14:44:14.719408 25885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:37595 every 8 connection(s)
I20260810 14:44:14.720685 25735 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:44:14.727036 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 25735
I20260810 14:44:14.727291 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:44:14.727861 25438 external_mini_cluster.cc:1428] Setting key 064bbea850a2b1ba2529021f0c09371d
Aug 10 14:44:14 dist-test-slave-vkjp krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373054, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:14.743934 25888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:14.723540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:40163 (local address 127.24.215.190:40989)
0810 14:44:14.724133 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:14.724137 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:14.725040 (+ 903us) server_negotiation.cc:407] Connection header received
0810 14:44:14.727050 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:14.727055 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:14.727218 (+ 163us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:14.727371 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:14.729326 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:14.730111 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:14.731800 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:14.732238 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:14.732378 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:14.736382 (+ 4004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:14.736392 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:14.736400 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:14.736448 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:14.740280 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:14.741059 (+ 779us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:14.741069 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:14.741075 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:14.741247 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:14.741695 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:14.741703 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:14.741709 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:14.742137 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:14.742310 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:14.743114 (+ 804us) server_negotiation.cc:299] Negotiation successful
0810 14:44:14.743397 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":329,"threads_started":1}
I20260810 14:44:14.744565 25889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:14.724065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:40989 (local address 127.24.215.130:40163)
0810 14:44:14.724876 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:14.724913 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:14.726723 (+ 1810us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:14.727664 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:14.727677 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:14.728474 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:14.729148 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:14.729169 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:14.730269 (+ 1100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:14.730275 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:14.731639 (+ 1364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:14.731651 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:14.732482 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:14.732489 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:14.732715 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:14.733411 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:14.733434 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:14.736136 (+ 2702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:14.740430 (+ 4294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:14.740438 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:14.740452 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:14.740910 (+ 458us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:14.741367 (+ 457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:14.741371 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:14.741373 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:14.741573 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:14.742299 (+ 726us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:14.742308 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:14.742687 (+ 379us) client_negotiation.cc:769] Sending connection context
0810 14:44:14.742942 (+ 255us) client_negotiation.cc:240] Negotiation successful
0810 14:44:14.743421 (+ 479us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":577,"thread_start_us":368,"threads_started":1}
I20260810 14:44:14.746188 25886 heartbeater.cc:347] Connected to a master server at 127.24.215.190:40989
I20260810 14:44:14.746578 25886 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
I20260810 14:44:14.747524 25886 heartbeater.cc:511] Master 127.24.215.190:40989 requested a full tablet report, sending...
I20260810 14:44:14.749017 25504 ts_manager.cc:194] Registered new tserver with Master: 14d8c2bba3804b78b591007daa217776 (127.24.215.130:37595)
I20260810 14:44:14.749843 25504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:40163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:44:14.774950 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:40989
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34797
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:15.118988 25892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:15.119424 25892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:15.119623 25892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:15.127178 25892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:15.127332 25892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:15.127542 25892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:44:15.137857 25892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34797
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:40989
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:15.139243 25892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:15.140697 25892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:15.155370 25899 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:15.155257 25903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:15.155470 25905 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:15.160607 25892 server_base.cc:1034] running on GCE node
I20260810 14:44:15.161293 25892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:15.162335 25892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:15.163539 25892 hybrid_clock.cc:648] HybridClock initialized: now 1786373055163479 us; error 45 us; skew 500 ppm
I20260810 14:44:15.163873 25892 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:15 dist-test-slave-vkjp krb5kdc[25448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373055, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:15.167745 25892 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:44:15.169416 25892 webserver.cc:460] Webserver started at http://127.24.215.131:42577/ using document root <none> and password file <none>
I20260810 14:44:15.170190 25892 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:15.170341 25892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:15.170683 25892 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:15.173821 25892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "2a641873047f497caad3b731eaa52d9c"
format_stamp: "Formatted at 2026-08-10 14:44:15 on dist-test-slave-vkjp"
server_key: "754582a58af656b8ac41144021c3a93a"
server_key_iv: "5a38d85a14c73aac59c57f5c38890cee"
server_key_version: "encryptionkey@0"
I20260810 14:44:15.174625 25892 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "2a641873047f497caad3b731eaa52d9c"
format_stamp: "Formatted at 2026-08-10 14:44:15 on dist-test-slave-vkjp"
server_key: "754582a58af656b8ac41144021c3a93a"
server_key_iv: "5a38d85a14c73aac59c57f5c38890cee"
server_key_version: "encryptionkey@0"
I20260810 14:44:15.180075 25892 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 14:44:15.183589 25914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:15.185030 25892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 14:44:15.185210 25892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "2a641873047f497caad3b731eaa52d9c"
format_stamp: "Formatted at 2026-08-10 14:44:15 on dist-test-slave-vkjp"
server_key: "754582a58af656b8ac41144021c3a93a"
server_key_iv: "5a38d85a14c73aac59c57f5c38890cee"
server_key_version: "encryptionkey@0"
I20260810 14:44:15.185425 25892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:15.200135 25892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:15.204640 25892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:15.204965 25892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:15.205991 25892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:15.207383 25892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:15.207491 25892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:15.207651 25892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:15.207752 25892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:15.221814 25729 heartbeater.cc:503] Master 127.24.215.190:40989 was elected leader, sending a full tablet report...
I20260810 14:44:15.245663 25892 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:43327
I20260810 14:44:15.245711 26046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:43327 every 8 connection(s)
I20260810 14:44:15.247036 25892 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:44:15.254141 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 25892
I20260810 14:44:15.254341 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:44:15.254850 25438 external_mini_cluster.cc:1428] Setting key 5f6fa88fa0dc7c92866b3e6a0be98310
Aug 10 14:44:15 dist-test-slave-vkjp krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373055, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:15.268867 26049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:15.249797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:37903 (local address 127.24.215.190:40989)
0810 14:44:15.250258 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:15.250262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:15.251369 (+ 1107us) server_negotiation.cc:407] Connection header received
0810 14:44:15.253232 (+ 1863us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:15.253237 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:15.253326 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:15.253427 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:15.255183 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:15.256076 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:15.258262 (+ 2186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:15.258814 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:15.258921 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:15.262838 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:15.262844 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:15.262848 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:15.262883 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:15.265754 (+ 2871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:15.266570 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:15.266574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:15.266576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:15.266716 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:15.267229 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:15.267233 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:15.267236 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:15.267547 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:15.267680 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:15.268373 (+ 693us) server_negotiation.cc:299] Negotiation successful
0810 14:44:15.268569 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":227,"threads_started":1}
I20260810 14:44:15.269857 26050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:15.250410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:40989 (local address 127.24.215.131:37903)
0810 14:44:15.251202 (+ 792us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:15.251241 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:15.252974 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:15.253644 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:15.253652 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:15.254238 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:15.254978 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:15.255000 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:15.256275 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:15.256281 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:15.258093 (+ 1812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:15.258110 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:15.259054 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:15.259060 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:15.259330 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:15.260131 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:15.260155 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:15.262595 (+ 2440us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:15.265928 (+ 3333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:15.265937 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:15.265959 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:15.266411 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:15.266818 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:15.266821 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:15.266823 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:15.267009 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:15.267662 (+ 653us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:15.267668 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:15.268006 (+ 338us) client_negotiation.cc:769] Sending connection context
0810 14:44:15.268256 (+ 250us) client_negotiation.cc:240] Negotiation successful
0810 14:44:15.268670 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":269,"threads_started":1}
I20260810 14:44:15.271442 26047 heartbeater.cc:347] Connected to a master server at 127.24.215.190:40989
I20260810 14:44:15.271780 26047 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:15.272562 26047 heartbeater.cc:511] Master 127.24.215.190:40989 requested a full tablet report, sending...
I20260810 14:44:15.274088 25504 ts_manager.cc:194] Registered new tserver with Master: 2a641873047f497caad3b731eaa52d9c (127.24.215.131:43327)
I20260810 14:44:15.274889 25504 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:37903
I20260810 14:44:15.284485 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:44:15 dist-test-slave-vkjp krb5kdc[25448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373055, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 14:44:15 dist-test-slave-vkjp krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373055, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:44:15.333915 26055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:15.317715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45238 (local address 127.24.215.129:42609)
0810 14:44:15.318477 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:15.318486 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:15.318522 (+ 36us) server_negotiation.cc:407] Connection header received
0810 14:44:15.318721 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:15.318729 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:15.318996 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:15.319198 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:15.320710 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:15.321745 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:15.323049 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:15.323705 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:15.323857 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:15.327918 (+ 4061us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:15.327930 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:15.327944 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:15.327978 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:15.330950 (+ 2972us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:15.331653 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:15.331661 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:15.331663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:15.331794 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:15.332211 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:15.332216 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:15.332221 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:15.332579 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:15.332896 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:15.333192 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 14:44:15.333501 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":536,"thread_start_us":361,"threads_started":1}
W20260810 14:44:15.335294 25631 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45238
Aug 10 14:44:15 dist-test-slave-vkjp krb5kdc[25448](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 14:44:15 dist-test-slave-vkjp krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:44:15.386802 26055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:15.371348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45246 (local address 127.24.215.129:42609)
0810 14:44:15.371676 (+ 328us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:15.371684 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:15.371937 (+ 253us) server_negotiation.cc:407] Connection header received
0810 14:44:15.372183 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:15.372193 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:15.372282 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:15.372407 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:15.374155 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:15.374960 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:15.376331 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:15.376879 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:15.377037 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:15.381413 (+ 4376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:15.381419 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:15.381423 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:15.381462 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:15.384199 (+ 2737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:15.384877 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:15.384883 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:15.384886 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:15.385025 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:15.385406 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:15.385415 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:15.385417 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:15.385667 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:15.385915 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:15.386241 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 14:44:15.386480 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131}
I20260810 14:44:15.388417 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 25579
I20260810 14:44:15.401631 26055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:15.389417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45248 (local address 127.24.215.129:42609)
0810 14:44:15.389636 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:15.389641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:15.389888 (+ 247us) server_negotiation.cc:407] Connection header received
0810 14:44:15.390077 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:15.390080 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:15.390134 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:15.390202 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:15.391391 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:15.392091 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:15.393286 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:15.393849 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:15.393944 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:15.396687 (+ 2743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:15.396692 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:15.396695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:15.396724 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:15.399215 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:15.399909 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:15.399913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:15.399916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:15.400068 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:15.400397 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:15.400401 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:15.400403 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:15.400654 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:15.400835 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:15.401074 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 14:44:15.401232 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 14:44:15.402441 25723 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:15.486581 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 25579
I20260810 14:44:15.505177 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 25735
Aug 10 14:44:15 dist-test-slave-vkjp krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:44:15.522703 26064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:15.506108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53204 (local address 127.24.215.130:37595)
0810 14:44:15.506679 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:15.506685 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:15.506714 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:44:15.506779 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:15.506787 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:15.506997 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:15.507223 (+ 226us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:15.508454 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:15.509560 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:15.510922 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:15.511536 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:15.511638 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:15.515822 (+ 4184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:15.515838 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:15.515854 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:15.515892 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:15.519329 (+ 3437us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:15.520123 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:15.520132 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:15.520135 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:15.520293 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:15.520794 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:15.520803 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:15.520806 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:15.521234 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:15.521596 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:15.521966 (+ 370us) server_negotiation.cc:299] Negotiation successful
0810 14:44:15.522305 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":413,"thread_start_us":293,"threads_started":1}
I20260810 14:44:15.524295 25881 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:15.618762 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 25735
I20260810 14:44:15.640542 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 25892
Aug 10 14:44:15 dist-test-slave-vkjp krb5kdc[25448](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373055, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:44:15.657613 26066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:15.641760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42334 (local address 127.24.215.131:43327)
0810 14:44:15.642451 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:15.642459 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:15.642489 (+ 30us) server_negotiation.cc:407] Connection header received
0810 14:44:15.642552 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:15.642558 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:15.642783 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:15.642978 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:15.644356 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:15.645428 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:15.646702 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:15.647346 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:15.647455 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:15.651807 (+ 4352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:15.651818 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:15.651833 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:15.651868 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:15.654617 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:15.655441 (+ 824us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:15.655448 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:15.655450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:15.655569 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:15.655973 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:15.655977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:15.655979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:15.656320 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:15.656648 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:15.656959 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 14:44:15.657241 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":506,"thread_start_us":342,"threads_started":1}
I20260810 14:44:15.658946 26042 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:15.742753 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 25892
I20260810 14:44:15.762739 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 25464
I20260810 14:44:15.764099 25546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:15.869032 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 25464
2026-08-10T14:44:15Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (4238 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:44:15 dist-test-slave-vkjp krb5kdc[26072](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:44:15 dist-test-slave-vkjp krb5kdc[26072](info): set up 2 sockets
Aug 10 14:44:15 dist-test-slave-vkjp krb5kdc[26072](info): commencing operation
krb5kdc: starting...
W20260810 14:44:17.041839 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.003s sys 0.014s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:44:17 dist-test-slave-vkjp krb5kdc[26072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:44:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:44:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:44:17.236148 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:37535
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:37535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:17.562021 26090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:17.562376 26090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:17.562554 26090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:17.569715 26090 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:17.569857 26090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:17.569969 26090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:17.570079 26090 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:17.570148 26090 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:44:17.579887 26090 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:37535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:37535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:17.581300 26090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:17.582628 26090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:17.594689 26097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:17.594920 26099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:17.595157 26096 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:17.596438 26090 server_base.cc:1034] running on GCE node
I20260810 14:44:17.597460 26090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:17.599328 26090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:17.600572 26090 hybrid_clock.cc:648] HybridClock initialized: now 1786373057600520 us; error 56 us; skew 500 ppm
I20260810 14:44:17.600926 26090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:17 dist-test-slave-vkjp krb5kdc[26072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373057, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:17.614048 26090 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:44:17.615751 26090 webserver.cc:460] Webserver started at http://127.24.215.190:41117/ using document root <none> and password file <none>
I20260810 14:44:17.616506 26090 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:17.616663 26090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:17.617017 26090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:17.620252 26090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "dc69cdabd7344651b02bbd8ff965e004"
format_stamp: "Formatted at 2026-08-10 14:44:17 on dist-test-slave-vkjp"
server_key: "fd25bf718fb08a7f2d32ff7668cc6981"
server_key_iv: "f9d4c7680527e5cb7116f7c71bdb6b1c"
server_key_version: "encryptionkey@0"
I20260810 14:44:17.621057 26090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "dc69cdabd7344651b02bbd8ff965e004"
format_stamp: "Formatted at 2026-08-10 14:44:17 on dist-test-slave-vkjp"
server_key: "fd25bf718fb08a7f2d32ff7668cc6981"
server_key_iv: "f9d4c7680527e5cb7116f7c71bdb6b1c"
server_key_version: "encryptionkey@0"
I20260810 14:44:17.626546 26090 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:44:17.630257 26110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:17.631809 26090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:44:17.632009 26090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "dc69cdabd7344651b02bbd8ff965e004"
format_stamp: "Formatted at 2026-08-10 14:44:17 on dist-test-slave-vkjp"
server_key: "fd25bf718fb08a7f2d32ff7668cc6981"
server_key_iv: "f9d4c7680527e5cb7116f7c71bdb6b1c"
server_key_version: "encryptionkey@0"
I20260810 14:44:17.632239 26090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:17.650578 26090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:17.655326 26090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:17.655660 26090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:17.674208 26090 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:37535
I20260810 14:44:17.674228 26170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:37535 every 8 connection(s)
I20260810 14:44:17.675683 26090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:44:17.679409 26171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:17.684993 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 26090
I20260810 14:44:17.685236 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:44:17.685887 25438 external_mini_cluster.cc:1428] Setting key d70f955ba59aa0550718d55c42e643ab
I20260810 14:44:17.691128 26171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004: Bootstrap starting.
I20260810 14:44:17.694995 26171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:17.696261 26171 log.cc:824] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:17.699666 26171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004: No bootstrap required, opened a new log
Aug 10 14:44:17 dist-test-slave-vkjp krb5kdc[26072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:17.708611 26171 raft_consensus.cc:348] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc69cdabd7344651b02bbd8ff965e004" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 37535 } }
I20260810 14:44:17.709072 26171 raft_consensus.cc:374] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:17.709192 26171 raft_consensus.cc:729] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc69cdabd7344651b02bbd8ff965e004, State: Initialized, Role: FOLLOWER
I20260810 14:44:17.709952 26171 consensus_queue.cc:260] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc69cdabd7344651b02bbd8ff965e004" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 37535 } }
I20260810 14:44:17.710237 26171 raft_consensus.cc:388] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:44:17.710407 26171 raft_consensus.cc:482] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:44:17.710602 26171 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:17.711551 26174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:17.687970 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54834 (local address 127.24.215.190:37535)
0810 14:44:17.688818 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:17.688833 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:17.688884 (+ 51us) server_negotiation.cc:407] Connection header received
0810 14:44:17.691318 (+ 2434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:17.691347 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:17.691848 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:17.692299 (+ 451us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:17.693732 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:17.695770 (+ 2038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:17.697546 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:17.698523 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:17.698711 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:17.703281 (+ 4570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:17.703289 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:17.703305 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:17.703357 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:17.706609 (+ 3252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:17.707524 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:17.707532 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:17.707538 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:17.707717 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:17.708199 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:17.708203 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:17.708205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:17.708806 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:17.709120 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:17.709740 (+ 620us) server_negotiation.cc:299] Negotiation successful
0810 14:44:17.710221 (+ 481us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":339,"threads_started":1}
I20260810 14:44:17.713203 26171 raft_consensus.cc:504] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc69cdabd7344651b02bbd8ff965e004" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 37535 } }
I20260810 14:44:17.713716 26171 leader_election.cc:302] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dc69cdabd7344651b02bbd8ff965e004; no voters:
I20260810 14:44:17.714376 26171 leader_election.cc:288] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:44:17.714524 26176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:17.715232 26176 raft_consensus.cc:686] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [term 1 LEADER]: Becoming Leader. State: Replica: dc69cdabd7344651b02bbd8ff965e004, State: Running, Role: LEADER
I20260810 14:44:17.715770 26176 consensus_queue.cc:237] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc69cdabd7344651b02bbd8ff965e004" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 37535 } }
I20260810 14:44:17.716723 26171 sys_catalog.cc:564] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:44:17.718580 26178 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dc69cdabd7344651b02bbd8ff965e004" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc69cdabd7344651b02bbd8ff965e004" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 37535 } } }
I20260810 14:44:17.718937 26178 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:17.719538 26179 sys_catalog.cc:455] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dc69cdabd7344651b02bbd8ff965e004. Latest consensus state: current_term: 1 leader_uuid: "dc69cdabd7344651b02bbd8ff965e004" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dc69cdabd7344651b02bbd8ff965e004" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 37535 } } }
I20260810 14:44:17.719902 26179 sys_catalog.cc:458] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004 [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:17.721805 26182 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:44:17.727010 26182 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:44:17.736752 26182 catalog_manager.cc:1347] Generated new cluster ID: 500bb6a9496f419ea0c9a31d0f3b769a
I20260810 14:44:17.736893 26182 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:44:17.751884 26182 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:44:17.753178 26182 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:44:17.765229 26182 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dc69cdabd7344651b02bbd8ff965e004: Generated new TSK 0
I20260810 14:44:17.766070 26182 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:44:17.817025 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:37535
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:18.166244 26199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:18.166555 26199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:18.166657 26199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:18.174013 26199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:18.174152 26199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:18.174273 26199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:44:18.184785 26199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:37535
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:18.186123 26199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:18.187613 26199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:18.201579 26204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:18.201598 26205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:18.201830 26207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:18.202793 26199 server_base.cc:1034] running on GCE node
I20260810 14:44:18.203528 26199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:18.204609 26199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:18.205847 26199 hybrid_clock.cc:648] HybridClock initialized: now 1786373058205802 us; error 62 us; skew 500 ppm
I20260810 14:44:18.206197 26199 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:18 dist-test-slave-vkjp krb5kdc[26072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373058, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:18.209527 26199 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:44:18.211242 26199 webserver.cc:460] Webserver started at http://127.24.215.129:39635/ using document root <none> and password file <none>
I20260810 14:44:18.211969 26199 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:18.212146 26199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:18.212512 26199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:18.215621 26199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "b62e58c37776467c966fbfcc5c45ad56"
format_stamp: "Formatted at 2026-08-10 14:44:18 on dist-test-slave-vkjp"
server_key: "90dd928785f0b14cd2374ce7f9d7da2b"
server_key_iv: "138abda8d1ddfe76a2d8cacfa24cf951"
server_key_version: "encryptionkey@0"
I20260810 14:44:18.216432 26199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "b62e58c37776467c966fbfcc5c45ad56"
format_stamp: "Formatted at 2026-08-10 14:44:18 on dist-test-slave-vkjp"
server_key: "90dd928785f0b14cd2374ce7f9d7da2b"
server_key_iv: "138abda8d1ddfe76a2d8cacfa24cf951"
server_key_version: "encryptionkey@0"
I20260810 14:44:18.222131 26199 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:44:18.225869 26215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:18.227387 26199 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:44:18.227563 26199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "b62e58c37776467c966fbfcc5c45ad56"
format_stamp: "Formatted at 2026-08-10 14:44:18 on dist-test-slave-vkjp"
server_key: "90dd928785f0b14cd2374ce7f9d7da2b"
server_key_iv: "138abda8d1ddfe76a2d8cacfa24cf951"
server_key_version: "encryptionkey@0"
I20260810 14:44:18.227787 26199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:18.242009 26199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:18.246650 26199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:18.246977 26199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:18.248024 26199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:18.249511 26199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:18.249583 26199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:18.249670 26199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:18.249715 26199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:18.283911 26199 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:46843
I20260810 14:44:18.283957 26346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:46843 every 8 connection(s)
I20260810 14:44:18.285265 26199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:44:18.290588 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 26199
I20260810 14:44:18.290813 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:44:18.291508 25438 external_mini_cluster.cc:1428] Setting key baf7b8adafda9b66f81d66cdd3fdf001
Aug 10 14:44:18 dist-test-slave-vkjp krb5kdc[26072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373058, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:18.308971 26349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:18.287882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:52469 (local address 127.24.215.190:37535)
0810 14:44:18.288338 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:18.288345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:18.289329 (+ 984us) server_negotiation.cc:407] Connection header received
0810 14:44:18.291412 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:18.291417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:18.291516 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:18.291638 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:18.293340 (+ 1702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:18.294193 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:18.296871 (+ 2678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:18.297307 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:18.297427 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:18.301424 (+ 3997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:18.301434 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:18.301443 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:18.301492 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:18.305366 (+ 3874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:18.306147 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:18.306156 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:18.306163 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:18.306324 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:18.306759 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:18.306768 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:18.306775 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:18.307240 (+ 465us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:18.307419 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:18.308165 (+ 746us) server_negotiation.cc:299] Negotiation successful
0810 14:44:18.308449 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":323,"thread_start_us":210,"threads_started":1}
I20260810 14:44:18.309553 26350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:18.288502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:37535 (local address 127.24.215.129:52469)
0810 14:44:18.289170 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:18.289209 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:18.291121 (+ 1912us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:18.291882 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:18.291890 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:18.292528 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:18.293105 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:18.293123 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:18.294371 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:18.294379 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:18.296671 (+ 2292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:18.296696 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:18.297548 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:18.297557 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:18.297903 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:18.298702 (+ 799us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:18.298741 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:18.301172 (+ 2431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:18.305523 (+ 4351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:18.305531 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:18.305545 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:18.305999 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:18.306436 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:18.306439 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:18.306441 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:18.306624 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:18.307381 (+ 757us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:18.307388 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:18.307731 (+ 343us) client_negotiation.cc:769] Sending connection context
0810 14:44:18.307973 (+ 242us) client_negotiation.cc:240] Negotiation successful
0810 14:44:18.308384 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":254,"threads_started":1}
I20260810 14:44:18.311188 26347 heartbeater.cc:347] Connected to a master server at 127.24.215.190:37535
I20260810 14:44:18.311561 26347 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
I20260810 14:44:18.312425 26347 heartbeater.cc:511] Master 127.24.215.190:37535 requested a full tablet report, sending...
I20260810 14:44:18.314682 26131 ts_manager.cc:194] Registered new tserver with Master: b62e58c37776467c966fbfcc5c45ad56 (127.24.215.129:46843)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
I20260810 14:44:18.316201 26131 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:52469
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:44:18.342411 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:37535
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:18.686690 26355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:18.687031 26355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:18.687245 26355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:18.694756 26355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:18.694918 26355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:18.695165 26355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:44:18.705336 26355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:37535
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:18.706733 26355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:18.708182 26355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:18.722183 26363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:18.722222 26360 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:18.722095 26361 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:18.723541 26355 server_base.cc:1034] running on GCE node
I20260810 14:44:18.724255 26355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:18.725245 26355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:18.726476 26355 hybrid_clock.cc:648] HybridClock initialized: now 1786373058726421 us; error 46 us; skew 500 ppm
I20260810 14:44:18.726809 26355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:18 dist-test-slave-vkjp krb5kdc[26072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373058, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:18.729945 26355 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:44:18.731667 26355 webserver.cc:460] Webserver started at http://127.24.215.130:41963/ using document root <none> and password file <none>
I20260810 14:44:18.732383 26355 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:18.732534 26355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:18.732908 26355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:18.736054 26355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "eb2508e0e254487386eb97c19f1750e4"
format_stamp: "Formatted at 2026-08-10 14:44:18 on dist-test-slave-vkjp"
server_key: "348b26d47cd4fbd8e4b0ea1e5ae62272"
server_key_iv: "c3be09465e74363e427500aa7451d74a"
server_key_version: "encryptionkey@0"
I20260810 14:44:18.736855 26355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "eb2508e0e254487386eb97c19f1750e4"
format_stamp: "Formatted at 2026-08-10 14:44:18 on dist-test-slave-vkjp"
server_key: "348b26d47cd4fbd8e4b0ea1e5ae62272"
server_key_iv: "c3be09465e74363e427500aa7451d74a"
server_key_version: "encryptionkey@0"
I20260810 14:44:18.742203 26355 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 14:44:18.745842 26372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:18.747201 26355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 14:44:18.747386 26355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "eb2508e0e254487386eb97c19f1750e4"
format_stamp: "Formatted at 2026-08-10 14:44:18 on dist-test-slave-vkjp"
server_key: "348b26d47cd4fbd8e4b0ea1e5ae62272"
server_key_iv: "c3be09465e74363e427500aa7451d74a"
server_key_version: "encryptionkey@0"
I20260810 14:44:18.747614 26355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:18.779870 26355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:18.784224 26355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:18.784543 26355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:18.785607 26355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:18.787408 26355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:18.787540 26355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:18.787693 26355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:18.787772 26355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:18.822505 26355 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:36195
I20260810 14:44:18.822530 26500 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:36195 every 8 connection(s)
I20260810 14:44:18.823899 26355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:44:18.825862 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 26355
I20260810 14:44:18.826092 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:44:18.826716 25438 external_mini_cluster.cc:1428] Setting key 1ea10cfe56fed1f2ce9ac03470cc0858
Aug 10 14:44:18 dist-test-slave-vkjp krb5kdc[26072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373058, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
I20260810 14:44:18.848078 26504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:18.826831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:59679 (local address 127.24.215.190:37535)
0810 14:44:18.827403 (+ 572us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:18.827409 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:18.830841 (+ 3432us) server_negotiation.cc:407] Connection header received
0810 14:44:18.831401 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:18.831409 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:18.831474 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:18.831566 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:18.833753 (+ 2187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:18.834623 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:18.836398 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:18.837291 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:18.837448 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:18.841578 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:18.841584 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:18.841587 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:18.841620 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:18.844613 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:18.845440 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:18.845445 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:18.845447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:18.845576 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:18.846079 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:18.846083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:18.846085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:18.846364 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:18.846527 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:18.847452 (+ 925us) server_negotiation.cc:299] Negotiation successful
0810 14:44:18.847693 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":307,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
I20260810 14:44:18.849131 26505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:18.827548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:37535 (local address 127.24.215.130:59679)
0810 14:44:18.828353 (+ 805us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:18.828398 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:18.831045 (+ 2647us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:18.831891 (+ 846us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:18.831906 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:18.832745 (+ 839us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:18.833554 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:18.833582 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:18.834804 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:18.834808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:18.836252 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:18.836265 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:18.837513 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:18.837518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:18.837719 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:18.838409 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:18.838437 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:18.841327 (+ 2890us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:18.844784 (+ 3457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:18.844794 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:18.844825 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:18.845276 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:18.845697 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:18.845702 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:18.845706 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:18.845937 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:18.846500 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:18.846508 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:18.846963 (+ 455us) client_negotiation.cc:769] Sending connection context
0810 14:44:18.847336 (+ 373us) client_negotiation.cc:240] Negotiation successful
0810 14:44:18.847903 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"thread_start_us":358,"threads_started":1}
I20260810 14:44:18.850903 26501 heartbeater.cc:347] Connected to a master server at 127.24.215.190:37535
I20260810 14:44:18.851315 26501 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:18.852149 26501 heartbeater.cc:511] Master 127.24.215.190:37535 requested a full tablet report, sending...
I20260810 14:44:18.853668 26131 ts_manager.cc:194] Registered new tserver with Master: eb2508e0e254487386eb97c19f1750e4 (127.24.215.130:36195)
I20260810 14:44:18.854504 26131 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:59679
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:44:18.874593 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:37535
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:19.210487 26508 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:19.210803 26508 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:19.210974 26508 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:19.218722 26508 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:19.218848 26508 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:19.219030 26508 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:44:19.229447 26508 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:37535
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:19.230952 26508 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:19.232671 26508 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:19.248464 26513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:19.248561 26514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:19.248522 26516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:19.249548 26508 server_base.cc:1034] running on GCE node
I20260810 14:44:19.250288 26508 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:19.251587 26508 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:19.252843 26508 hybrid_clock.cc:648] HybridClock initialized: now 1786373059252789 us; error 45 us; skew 500 ppm
I20260810 14:44:19.253196 26508 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:19 dist-test-slave-vkjp krb5kdc[26072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373059, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:19.256749 26508 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:44:19.258268 26508 webserver.cc:460] Webserver started at http://127.24.215.131:39909/ using document root <none> and password file <none>
I20260810 14:44:19.258960 26508 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:19.259069 26508 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:19.259485 26508 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:19.262550 26508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "9c996344213c4fef9b9feeb3fcb4314b"
format_stamp: "Formatted at 2026-08-10 14:44:19 on dist-test-slave-vkjp"
server_key: "16d43e83c6b09b3a041765f968fbb33b"
server_key_iv: "29e2304ece03c44478eb7f97348309a9"
server_key_version: "encryptionkey@0"
I20260810 14:44:19.263473 26508 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "9c996344213c4fef9b9feeb3fcb4314b"
format_stamp: "Formatted at 2026-08-10 14:44:19 on dist-test-slave-vkjp"
server_key: "16d43e83c6b09b3a041765f968fbb33b"
server_key_iv: "29e2304ece03c44478eb7f97348309a9"
server_key_version: "encryptionkey@0"
I20260810 14:44:19.269016 26508 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 14:44:19.272723 26524 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:19.274232 26508 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 14:44:19.274394 26508 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "9c996344213c4fef9b9feeb3fcb4314b"
format_stamp: "Formatted at 2026-08-10 14:44:19 on dist-test-slave-vkjp"
server_key: "16d43e83c6b09b3a041765f968fbb33b"
server_key_iv: "29e2304ece03c44478eb7f97348309a9"
server_key_version: "encryptionkey@0"
I20260810 14:44:19.274573 26508 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:19.291821 26508 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:19.296180 26508 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:19.296443 26508 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:19.297427 26508 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:19.298786 26508 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:19.298862 26508 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:19.298954 26508 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:19.298996 26508 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:19.319463 26347 heartbeater.cc:503] Master 127.24.215.190:37535 was elected leader, sending a full tablet report...
I20260810 14:44:19.333529 26508 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:44457
I20260810 14:44:19.333576 26652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:44457 every 8 connection(s)
I20260810 14:44:19.334888 26508 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:44:19.345012 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 26508
I20260810 14:44:19.345220 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:44:19.345753 25438 external_mini_cluster.cc:1428] Setting key 3cfe14a9ec9ab1102e3d4fd342d19911
Aug 10 14:44:19 dist-test-slave-vkjp krb5kdc[26072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373059, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:19.357075 26504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.337762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:49163 (local address 127.24.215.190:37535)
0810 14:44:19.337970 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:19.337975 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:19.339294 (+ 1319us) server_negotiation.cc:407] Connection header received
0810 14:44:19.341295 (+ 2001us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:19.341300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:19.341388 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:19.341491 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:19.343076 (+ 1585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.343995 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.346201 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.346824 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.346950 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.351600 (+ 4650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:19.351605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:19.351608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:19.351662 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:19.354189 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:19.354928 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:19.354933 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:19.354935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:19.355062 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:19.355506 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:19.355509 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:19.355511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:19.355807 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:19.355943 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:19.356642 (+ 699us) server_negotiation.cc:299] Negotiation successful
0810 14:44:19.356825 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 14:44:19.358089 26656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.338303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:37535 (local address 127.24.215.131:49163)
0810 14:44:19.339065 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:19.339164 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:19.340997 (+ 1833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:19.341700 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:19.341708 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:19.342246 (+ 538us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:19.342879 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:19.342897 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.344164 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.344168 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:19.346002 (+ 1834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:19.346017 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.347375 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.347386 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:19.347640 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.348519 (+ 879us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:19.348555 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:19.351335 (+ 2780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:19.354324 (+ 2989us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:19.354332 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:19.354346 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:19.354803 (+ 457us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:19.355211 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:19.355214 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:19.355217 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:19.355406 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:19.355919 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:19.355925 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:19.356272 (+ 347us) client_negotiation.cc:769] Sending connection context
0810 14:44:19.356534 (+ 262us) client_negotiation.cc:240] Negotiation successful
0810 14:44:19.356938 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":316,"threads_started":1}
I20260810 14:44:19.359601 26653 heartbeater.cc:347] Connected to a master server at 127.24.215.190:37535
I20260810 14:44:19.359952 26653 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:19.360740 26653 heartbeater.cc:511] Master 127.24.215.190:37535 requested a full tablet report, sending...
I20260810 14:44:19.362432 26131 ts_manager.cc:194] Registered new tserver with Master: 9c996344213c4fef9b9feeb3fcb4314b (127.24.215.131:44457)
I20260810 14:44:19.363281 26131 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:49163
I20260810 14:44:19.374632 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:44:19.398000 26504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.381499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54844 (local address 127.24.215.190:37535)
0810 14:44:19.381820 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:19.381828 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:19.382212 (+ 384us) server_negotiation.cc:407] Connection header received
0810 14:44:19.382430 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:19.382436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:19.382523 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:19.382634 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:19.384945 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.386150 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.388403 (+ 2253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.389299 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.389436 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.392423 (+ 2987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:19.392429 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:19.392432 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:19.392468 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:19.395364 (+ 2896us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:19.396032 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:19.396036 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:19.396038 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:19.396171 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:19.396521 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:19.396524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:19.396526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:19.396862 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:19.397051 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:19.397423 (+ 372us) server_negotiation.cc:299] Negotiation successful
0810 14:44:19.397676 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 14:44:19.404623 26131 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54844:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:44:19.411737 26131 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:44:19.437381 26668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.425605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55158 (local address 127.24.215.130:36195)
0810 14:44:19.426436 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:19.426448 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:19.427001 (+ 553us) server_negotiation.cc:407] Connection header received
0810 14:44:19.427442 (+ 441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:19.427450 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:19.427708 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:19.427877 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:19.429117 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.430812 (+ 1695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.434188 (+ 3376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.435770 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.435908 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.436091 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:19.436879 (+ 788us) server_negotiation.cc:299] Negotiation successful
0810 14:44:19.437044 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":516,"threads_started":1}
I20260810 14:44:19.437532 26670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.425892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:44457 (local address 127.0.0.1:57382)
0810 14:44:19.427069 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:19.427113 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:19.427280 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:19.428422 (+ 1142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:19.428428 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:19.428478 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:19.429176 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:19.429190 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.431258 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.431266 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:19.433997 (+ 2731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:19.434013 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.436321 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.436334 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:19.436559 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.436653 (+ 94us) client_negotiation.cc:769] Sending connection context
0810 14:44:19.436800 (+ 147us) client_negotiation.cc:240] Negotiation successful
0810 14:44:19.437016 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1010,"mutex_wait_us":160,"spinlock_wait_cycles":14208,"thread_start_us":1172,"threads_started":1}
I20260810 14:44:19.438293 26669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.425322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:36195 (local address 127.0.0.1:55158)
0810 14:44:19.426855 (+ 1533us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:19.426890 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:19.427184 (+ 294us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:19.428185 (+ 1001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:19.428192 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:19.428234 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:19.428828 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:19.428848 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.431055 (+ 2207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.431060 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:19.433997 (+ 2937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:19.434013 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.436347 (+ 2334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.436352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:19.436645 (+ 293us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.436670 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 14:44:19.437689 (+ 1019us) client_negotiation.cc:240] Negotiation successful
0810 14:44:19.437830 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1300,"mutex_wait_us":144,"thread_start_us":1132,"threads_started":1}
I20260810 14:44:19.438938 26671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.426085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57382 (local address 127.24.215.131:44457)
0810 14:44:19.427890 (+ 1805us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:19.427920 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:19.427952 (+ 32us) server_negotiation.cc:407] Connection header received
0810 14:44:19.428049 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:19.428054 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:19.428200 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:19.428411 (+ 211us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:19.429400 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.431099 (+ 1699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.434188 (+ 3089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.435712 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.435913 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.436091 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:19.438291 (+ 2200us) server_negotiation.cc:299] Negotiation successful
0810 14:44:19.438475 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1633,"thread_start_us":1479,"threads_started":1}
I20260810 14:44:19.443142 26579 tablet_service.cc:1467] Processing CreateTablet for tablet 4388457a6fe8452c8b630663ff4748f3 (DEFAULT_TABLE table=test-table [id=e8cbc439866246a89466be0b349b9275]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:19.444906 26579 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4388457a6fe8452c8b630663ff4748f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:19.443307 26428 tablet_service.cc:1467] Processing CreateTablet for tablet 4388457a6fe8452c8b630663ff4748f3 (DEFAULT_TABLE table=test-table [id=e8cbc439866246a89466be0b349b9275]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:19.446393 26428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4388457a6fe8452c8b630663ff4748f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:19.450133 26670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.428997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46843 (local address 127.0.0.1:47700)
0810 14:44:19.437738 (+ 8741us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:19.437757 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:19.437851 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:19.441961 (+ 4110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:19.441967 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:19.442010 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:19.442200 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:19.442211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.444989 (+ 2778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.444995 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:19.447024 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:19.447039 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.449135 (+ 2096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.449144 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:19.449438 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.449482 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 14:44:19.449614 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 14:44:19.449783 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8643,"thread_start_us":15456,"threads_started":1}
I20260810 14:44:19.450169 26673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.429421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47700 (local address 127.24.215.129:46843)
0810 14:44:19.441356 (+ 11935us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:19.441364 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:19.441394 (+ 30us) server_negotiation.cc:407] Connection header received
0810 14:44:19.441490 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:19.441497 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:19.441741 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:19.442389 (+ 648us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:19.442626 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.444171 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.447402 (+ 3231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.448919 (+ 1517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.449136 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.449312 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:19.449654 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 14:44:19.449826 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11732,"thread_start_us":19878,"threads_started":1}
I20260810 14:44:19.454494 26273 tablet_service.cc:1467] Processing CreateTablet for tablet 4388457a6fe8452c8b630663ff4748f3 (DEFAULT_TABLE table=test-table [id=e8cbc439866246a89466be0b349b9275]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:19.456022 26273 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4388457a6fe8452c8b630663ff4748f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:19.459880 26674 tablet_bootstrap.cc:492] T 4388457a6fe8452c8b630663ff4748f3 P 9c996344213c4fef9b9feeb3fcb4314b: Bootstrap starting.
I20260810 14:44:19.462702 26674 tablet_bootstrap.cc:654] T 4388457a6fe8452c8b630663ff4748f3 P 9c996344213c4fef9b9feeb3fcb4314b: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:19.463811 26674 log.cc:824] T 4388457a6fe8452c8b630663ff4748f3 P 9c996344213c4fef9b9feeb3fcb4314b: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:19.465009 26675 tablet_bootstrap.cc:492] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4: Bootstrap starting.
I20260810 14:44:19.469097 26675 tablet_bootstrap.cc:654] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:19.469189 26674 tablet_bootstrap.cc:492] T 4388457a6fe8452c8b630663ff4748f3 P 9c996344213c4fef9b9feeb3fcb4314b: No bootstrap required, opened a new log
I20260810 14:44:19.469494 26674 ts_tablet_manager.cc:1397] T 4388457a6fe8452c8b630663ff4748f3 P 9c996344213c4fef9b9feeb3fcb4314b: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.006s
I20260810 14:44:19.470575 26675 log.cc:824] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:19.471585 26677 tablet_bootstrap.cc:492] T 4388457a6fe8452c8b630663ff4748f3 P b62e58c37776467c966fbfcc5c45ad56: Bootstrap starting.
I20260810 14:44:19.475522 26677 tablet_bootstrap.cc:654] T 4388457a6fe8452c8b630663ff4748f3 P b62e58c37776467c966fbfcc5c45ad56: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:19.475816 26675 tablet_bootstrap.cc:492] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4: No bootstrap required, opened a new log
I20260810 14:44:19.476145 26675 ts_tablet_manager.cc:1397] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260810 14:44:19.476994 26677 log.cc:824] T 4388457a6fe8452c8b630663ff4748f3 P b62e58c37776467c966fbfcc5c45ad56: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:19.480674 26677 tablet_bootstrap.cc:492] T 4388457a6fe8452c8b630663ff4748f3 P b62e58c37776467c966fbfcc5c45ad56: No bootstrap required, opened a new log
I20260810 14:44:19.480978 26677 ts_tablet_manager.cc:1397] T 4388457a6fe8452c8b630663ff4748f3 P b62e58c37776467c966fbfcc5c45ad56: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 14:44:19.480875 26674 raft_consensus.cc:348] T 4388457a6fe8452c8b630663ff4748f3 P 9c996344213c4fef9b9feeb3fcb4314b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2508e0e254487386eb97c19f1750e4" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36195 } } peers { permanent_uuid: "9c996344213c4fef9b9feeb3fcb4314b" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 44457 } } peers { permanent_uuid: "b62e58c37776467c966fbfcc5c45ad56" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46843 } }
I20260810 14:44:19.481432 26674 raft_consensus.cc:374] T 4388457a6fe8452c8b630663ff4748f3 P 9c996344213c4fef9b9feeb3fcb4314b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:19.481556 26674 raft_consensus.cc:729] T 4388457a6fe8452c8b630663ff4748f3 P 9c996344213c4fef9b9feeb3fcb4314b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c996344213c4fef9b9feeb3fcb4314b, State: Initialized, Role: FOLLOWER
I20260810 14:44:19.482260 26674 consensus_queue.cc:260] T 4388457a6fe8452c8b630663ff4748f3 P 9c996344213c4fef9b9feeb3fcb4314b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2508e0e254487386eb97c19f1750e4" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36195 } } peers { permanent_uuid: "9c996344213c4fef9b9feeb3fcb4314b" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 44457 } } peers { permanent_uuid: "b62e58c37776467c966fbfcc5c45ad56" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46843 } }
I20260810 14:44:19.483821 26653 heartbeater.cc:503] Master 127.24.215.190:37535 was elected leader, sending a full tablet report...
I20260810 14:44:19.486052 26674 ts_tablet_manager.cc:1428] T 4388457a6fe8452c8b630663ff4748f3 P 9c996344213c4fef9b9feeb3fcb4314b: Time spent starting tablet: real 0.016s user 0.015s sys 0.000s
I20260810 14:44:19.487483 26675 raft_consensus.cc:348] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2508e0e254487386eb97c19f1750e4" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36195 } } peers { permanent_uuid: "9c996344213c4fef9b9feeb3fcb4314b" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 44457 } } peers { permanent_uuid: "b62e58c37776467c966fbfcc5c45ad56" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46843 } }
I20260810 14:44:19.487913 26675 raft_consensus.cc:374] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:19.488040 26675 raft_consensus.cc:729] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb2508e0e254487386eb97c19f1750e4, State: Initialized, Role: FOLLOWER
I20260810 14:44:19.488683 26675 consensus_queue.cc:260] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2508e0e254487386eb97c19f1750e4" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36195 } } peers { permanent_uuid: "9c996344213c4fef9b9feeb3fcb4314b" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 44457 } } peers { permanent_uuid: "b62e58c37776467c966fbfcc5c45ad56" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46843 } }
I20260810 14:44:19.489913 26501 heartbeater.cc:503] Master 127.24.215.190:37535 was elected leader, sending a full tablet report...
I20260810 14:44:19.491816 26677 raft_consensus.cc:348] T 4388457a6fe8452c8b630663ff4748f3 P b62e58c37776467c966fbfcc5c45ad56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2508e0e254487386eb97c19f1750e4" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36195 } } peers { permanent_uuid: "9c996344213c4fef9b9feeb3fcb4314b" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 44457 } } peers { permanent_uuid: "b62e58c37776467c966fbfcc5c45ad56" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46843 } }
I20260810 14:44:19.492362 26677 raft_consensus.cc:374] T 4388457a6fe8452c8b630663ff4748f3 P b62e58c37776467c966fbfcc5c45ad56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:19.492480 26677 raft_consensus.cc:729] T 4388457a6fe8452c8b630663ff4748f3 P b62e58c37776467c966fbfcc5c45ad56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b62e58c37776467c966fbfcc5c45ad56, State: Initialized, Role: FOLLOWER
I20260810 14:44:19.493342 26675 ts_tablet_manager.cc:1428] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4: Time spent starting tablet: real 0.016s user 0.010s sys 0.005s
I20260810 14:44:19.493324 26677 consensus_queue.cc:260] T 4388457a6fe8452c8b630663ff4748f3 P b62e58c37776467c966fbfcc5c45ad56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2508e0e254487386eb97c19f1750e4" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36195 } } peers { permanent_uuid: "9c996344213c4fef9b9feeb3fcb4314b" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 44457 } } peers { permanent_uuid: "b62e58c37776467c966fbfcc5c45ad56" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46843 } }
I20260810 14:44:19.495767 26677 ts_tablet_manager.cc:1428] T 4388457a6fe8452c8b630663ff4748f3 P b62e58c37776467c966fbfcc5c45ad56: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
W20260810 14:44:19.537678 26348 tablet.cc:2369] T 4388457a6fe8452c8b630663ff4748f3 P b62e58c37776467c966fbfcc5c45ad56: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:44:19.576424 26502 tablet.cc:2369] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:44:19.586724 26655 tablet.cc:2369] T 4388457a6fe8452c8b630663ff4748f3 P 9c996344213c4fef9b9feeb3fcb4314b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:44:19.728668 26681 raft_consensus.cc:482] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:44:19.728962 26681 raft_consensus.cc:504] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2508e0e254487386eb97c19f1750e4" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36195 } } peers { permanent_uuid: "9c996344213c4fef9b9feeb3fcb4314b" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 44457 } } peers { permanent_uuid: "b62e58c37776467c966fbfcc5c45ad56" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46843 } }
I20260810 14:44:19.730908 26681 leader_election.cc:288] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9c996344213c4fef9b9feeb3fcb4314b (127.24.215.131:44457), b62e58c37776467c966fbfcc5c45ad56 (127.24.215.129:46843)
I20260810 14:44:19.740347 26671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.731576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:32941 (local address 127.24.215.131:44457)
0810 14:44:19.732033 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:19.732042 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:19.732673 (+ 631us) server_negotiation.cc:407] Connection header received
0810 14:44:19.732975 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:19.732983 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:19.733102 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:19.733244 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:19.734003 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.735405 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.737960 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.739429 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.739556 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.739611 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:19.740035 (+ 424us) server_negotiation.cc:299] Negotiation successful
0810 14:44:19.740129 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263}
I20260810 14:44:19.740585 26683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.731327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:44457 (local address 127.24.215.130:32941)
0810 14:44:19.732531 (+ 1204us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:19.732553 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:19.732774 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:19.733311 (+ 537us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:19.733316 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:19.733352 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:19.733807 (+ 455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:19.733824 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.735573 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.735579 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:19.737764 (+ 2185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:19.737781 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.739671 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.739677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:19.739846 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.739895 (+ 49us) client_negotiation.cc:769] Sending connection context
0810 14:44:19.739992 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 14:44:19.740126 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1049,"mutex_wait_us":1,"thread_start_us":794,"threads_started":1}
I20260810 14:44:19.741691 26673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.731768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:48995 (local address 127.24.215.129:46843)
0810 14:44:19.732176 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:19.732180 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:19.733654 (+ 1474us) server_negotiation.cc:407] Connection header received
0810 14:44:19.733842 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:19.733851 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:19.733960 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:19.734064 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:19.734888 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.736530 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.739249 (+ 2719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.740414 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.740587 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.740670 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:19.741305 (+ 635us) server_negotiation.cc:299] Negotiation successful
0810 14:44:19.741456 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":289}
I20260810 14:44:19.741842 26684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.731648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46843 (local address 127.24.215.130:48995)
0810 14:44:19.733552 (+ 1904us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:19.733566 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:19.733668 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:19.734110 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:19.734114 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:19.734161 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:19.734692 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:19.734704 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.736779 (+ 2075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.736783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:19.738969 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:19.738991 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.740941 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.740948 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:19.741140 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.741164 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 14:44:19.741244 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 14:44:19.741333 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1748,"thread_start_us":1531,"threads_started":1}
I20260810 14:44:19.741730 26605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4388457a6fe8452c8b630663ff4748f3" candidate_uuid: "eb2508e0e254487386eb97c19f1750e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c996344213c4fef9b9feeb3fcb4314b" is_pre_election: true
I20260810 14:44:19.742314 26605 raft_consensus.cc:2454] T 4388457a6fe8452c8b630663ff4748f3 P 9c996344213c4fef9b9feeb3fcb4314b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb2508e0e254487386eb97c19f1750e4 in term 0.
I20260810 14:44:19.742928 26295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4388457a6fe8452c8b630663ff4748f3" candidate_uuid: "eb2508e0e254487386eb97c19f1750e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b62e58c37776467c966fbfcc5c45ad56" is_pre_election: true
I20260810 14:44:19.743148 26373 leader_election.cc:302] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9c996344213c4fef9b9feeb3fcb4314b, eb2508e0e254487386eb97c19f1750e4; no voters:
I20260810 14:44:19.743532 26295 raft_consensus.cc:2454] T 4388457a6fe8452c8b630663ff4748f3 P b62e58c37776467c966fbfcc5c45ad56 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb2508e0e254487386eb97c19f1750e4 in term 0.
I20260810 14:44:19.743575 26681 raft_consensus.cc:2781] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:44:19.743815 26681 raft_consensus.cc:482] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:44:19.743947 26681 raft_consensus.cc:3037] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:19.746292 26681 raft_consensus.cc:504] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2508e0e254487386eb97c19f1750e4" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36195 } } peers { permanent_uuid: "9c996344213c4fef9b9feeb3fcb4314b" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 44457 } } peers { permanent_uuid: "b62e58c37776467c966fbfcc5c45ad56" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46843 } }
I20260810 14:44:19.746927 26681 leader_election.cc:288] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [CANDIDATE]: Term 1 election: Requested vote from peers 9c996344213c4fef9b9feeb3fcb4314b (127.24.215.131:44457), b62e58c37776467c966fbfcc5c45ad56 (127.24.215.129:46843)
I20260810 14:44:19.747479 26605 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4388457a6fe8452c8b630663ff4748f3" candidate_uuid: "eb2508e0e254487386eb97c19f1750e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c996344213c4fef9b9feeb3fcb4314b"
I20260810 14:44:19.747464 26295 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4388457a6fe8452c8b630663ff4748f3" candidate_uuid: "eb2508e0e254487386eb97c19f1750e4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b62e58c37776467c966fbfcc5c45ad56"
I20260810 14:44:19.747690 26295 raft_consensus.cc:3037] T 4388457a6fe8452c8b630663ff4748f3 P b62e58c37776467c966fbfcc5c45ad56 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:19.747690 26605 raft_consensus.cc:3037] T 4388457a6fe8452c8b630663ff4748f3 P 9c996344213c4fef9b9feeb3fcb4314b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:19.751009 26605 raft_consensus.cc:2454] T 4388457a6fe8452c8b630663ff4748f3 P 9c996344213c4fef9b9feeb3fcb4314b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb2508e0e254487386eb97c19f1750e4 in term 1.
I20260810 14:44:19.751114 26295 raft_consensus.cc:2454] T 4388457a6fe8452c8b630663ff4748f3 P b62e58c37776467c966fbfcc5c45ad56 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb2508e0e254487386eb97c19f1750e4 in term 1.
I20260810 14:44:19.751595 26374 leader_election.cc:302] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b62e58c37776467c966fbfcc5c45ad56, eb2508e0e254487386eb97c19f1750e4; no voters:
I20260810 14:44:19.751978 26681 raft_consensus.cc:2781] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:19.752568 26681 raft_consensus.cc:686] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [term 1 LEADER]: Becoming Leader. State: Replica: eb2508e0e254487386eb97c19f1750e4, State: Running, Role: LEADER
I20260810 14:44:19.753166 26681 consensus_queue.cc:237] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2508e0e254487386eb97c19f1750e4" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36195 } } peers { permanent_uuid: "9c996344213c4fef9b9feeb3fcb4314b" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 44457 } } peers { permanent_uuid: "b62e58c37776467c966fbfcc5c45ad56" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46843 } }
I20260810 14:44:19.759418 26129 catalog_manager.cc:5495] T 4388457a6fe8452c8b630663ff4748f3 P eb2508e0e254487386eb97c19f1750e4 reported cstate change: term changed from 0 to 1, leader changed from <none> to eb2508e0e254487386eb97c19f1750e4 (127.24.215.130). New cstate: current_term: 1 leader_uuid: "eb2508e0e254487386eb97c19f1750e4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eb2508e0e254487386eb97c19f1750e4" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36195 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9c996344213c4fef9b9feeb3fcb4314b" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 44457 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b62e58c37776467c966fbfcc5c45ad56" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46843 } health_report { overall_health: UNKNOWN } } }
Aug 10 14:44:19 dist-test-slave-vkjp krb5kdc[26072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373057, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:44:19.814656 26673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.799288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47710 (local address 127.24.215.129:46843)
0810 14:44:19.799492 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:19.799500 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:19.799997 (+ 497us) server_negotiation.cc:407] Connection header received
0810 14:44:19.800216 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:19.800222 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:19.800299 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:19.800408 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:19.802061 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.802920 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.804246 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.804673 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.804813 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.808955 (+ 4142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:19.808972 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:19.808990 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:19.809031 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:19.811960 (+ 2929us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:19.812680 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:19.812688 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:19.812690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:19.812845 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:19.813225 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:19.813229 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:19.813231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:19.813612 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:19.813856 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:19.814202 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 14:44:19.814379 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
Aug 10 14:44:19 dist-test-slave-vkjp krb5kdc[26072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373059, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 14:44:19 dist-test-slave-vkjp krb5kdc[26072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373059, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:44:19.865837 26673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.851644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47722 (local address 127.24.215.129:46843)
0810 14:44:19.851912 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:19.851921 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:19.852275 (+ 354us) server_negotiation.cc:407] Connection header received
0810 14:44:19.852390 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:19.852394 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:19.852461 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:19.852547 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:19.854036 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.854776 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.856084 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.856542 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.856652 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.860818 (+ 4166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:19.860824 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:19.860828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:19.860863 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:19.863607 (+ 2744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:19.864221 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:19.864224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:19.864227 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:19.864352 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:19.864672 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:19.864675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:19.864677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:19.864927 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:19.865141 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:19.865396 (+ 255us) server_negotiation.cc:299] Negotiation successful
0810 14:44:19.865570 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
Aug 10 14:44:19 dist-test-slave-vkjp krb5kdc[26072](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 14:44:19 dist-test-slave-vkjp krb5kdc[26072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:44:19.915220 26673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.900025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47726 (local address 127.24.215.129:46843)
0810 14:44:19.900244 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:19.900253 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:19.900557 (+ 304us) server_negotiation.cc:407] Connection header received
0810 14:44:19.900780 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:19.900785 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:19.900852 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:19.900958 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:19.902284 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.903289 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.904834 (+ 1545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.905258 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.905359 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.909540 (+ 4181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:19.909546 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:19.909549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:19.909584 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:19.912462 (+ 2878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:19.913196 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:19.913200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:19.913202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:19.913319 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:19.913662 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:19.913665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:19.913667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:19.913991 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:19.914273 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:19.914612 (+ 339us) server_negotiation.cc:299] Negotiation successful
0810 14:44:19.914783 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260810 14:44:19.917089 26249 tablet_service.cc:2908] Rejecting scan request for tablet 4388457a6fe8452c8b630663ff4748f3: Uninitialized: clean time has not yet been initialized
I20260810 14:44:19.920176 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 26199
I20260810 14:44:19.933964 26673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:19.921190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47732 (local address 127.24.215.129:46843)
0810 14:44:19.921410 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:19.921425 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:19.921627 (+ 202us) server_negotiation.cc:407] Connection header received
0810 14:44:19.921761 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:19.921764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:19.921833 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:19.921914 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:19.923262 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.924008 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.925281 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:19.925767 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:19.925874 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:19.928535 (+ 2661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:19.928540 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:19.928544 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:19.928576 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:19.931382 (+ 2806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:19.932087 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:19.932091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:19.932093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:19.932210 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:19.932644 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:19.932647 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:19.932649 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:19.932954 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:19.933163 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:19.933453 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 14:44:19.933618 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:44:19.934676 26340 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:20.026476 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 26199
I20260810 14:44:20.048460 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 26355
Aug 10 14:44:20 dist-test-slave-vkjp krb5kdc[26072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:44:20.064917 26700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:20.049258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55170 (local address 127.24.215.130:36195)
0810 14:44:20.049724 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:20.049728 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:20.049745 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:44:20.049801 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:20.049805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:20.049879 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:20.049973 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:20.051288 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:20.052184 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:20.053693 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:20.054104 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:20.054212 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:20.058296 (+ 4084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:20.058312 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:20.058329 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:20.058367 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:20.062031 (+ 3664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:20.062796 (+ 765us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:20.062804 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:20.062806 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:20.062963 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:20.063418 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:20.063422 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:20.063424 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:20.063777 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:20.064011 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:20.064311 (+ 300us) server_negotiation.cc:299] Negotiation successful
0810 14:44:20.064506 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":229,"threads_started":1}
I20260810 14:44:20.065634 26496 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:20.157815 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 26355
I20260810 14:44:20.180088 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 26508
Aug 10 14:44:20 dist-test-slave-vkjp krb5kdc[26072](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373059, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:44:20.195894 26671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:20.180822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57398 (local address 127.24.215.131:44457)
0810 14:44:20.181017 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:20.181035 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:20.181134 (+ 99us) server_negotiation.cc:407] Connection header received
0810 14:44:20.181363 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:20.181368 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:20.181435 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:20.181546 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:20.182780 (+ 1234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:20.183569 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:20.184894 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:20.185274 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:20.185409 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:20.189864 (+ 4455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:20.189881 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:20.189898 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:20.189948 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:20.192814 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:20.193537 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:20.193545 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:20.193548 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:20.193705 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:20.194177 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:20.194182 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:20.194184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:20.194622 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:20.194881 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:20.195419 (+ 538us) server_negotiation.cc:299] Negotiation successful
0810 14:44:20.195597 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:44:20.196660 26648 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:20.288003 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 26508
I20260810 14:44:20.309559 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 26090
I20260810 14:44:20.310824 26165 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:20.420989 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 26090
2026-08-10T14:44:20Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4553 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:44:20 dist-test-slave-vkjp krb5kdc[26706](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:44:20 dist-test-slave-vkjp krb5kdc[26706](info): set up 2 sockets
Aug 10 14:44:20 dist-test-slave-vkjp krb5kdc[26706](info): commencing operation
krb5kdc: starting...
W20260810 14:44:21.605602 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.000s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:44:21 dist-test-slave-vkjp krb5kdc[26706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:44:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:44:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:44:21.802881 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:40341
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:40341
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:22.131759 26721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:22.132104 26721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:22.132267 26721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:22.140164 26721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:22.140305 26721 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 14:44:22.140407 26721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:22.140530 26721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:22.140615 26721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:22.140708 26721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:44:22.150781 26721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:40341
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:40341
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:22.152158 26721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:22.153528 26721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:22.165884 26727 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:22.165972 26733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:22.165920 26729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:22.166990 26721 server_base.cc:1034] running on GCE node
I20260810 14:44:22.167807 26721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:22.169178 26721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:22.170444 26721 hybrid_clock.cc:648] HybridClock initialized: now 1786373062170367 us; error 42 us; skew 500 ppm
I20260810 14:44:22.170825 26721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:22 dist-test-slave-vkjp krb5kdc[26706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373062, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:22.174301 26721 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:44:22.175966 26721 webserver.cc:460] Webserver started at http://127.24.215.190:32843/ using document root <none> and password file <none>
I20260810 14:44:22.176638 26721 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:22.176723 26721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:22.177007 26721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:22.179960 26721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "8f6e26ec58524e3f86bc564ddfd0e893"
format_stamp: "Formatted at 2026-08-10 14:44:22 on dist-test-slave-vkjp"
server_key: "2e1dbef8c27e8367bafcd0c9c74878f2"
server_key_iv: "cc854e1c1c8028dda75237fc1676afa7"
server_key_version: "encryptionkey@0"
I20260810 14:44:22.180673 26721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "8f6e26ec58524e3f86bc564ddfd0e893"
format_stamp: "Formatted at 2026-08-10 14:44:22 on dist-test-slave-vkjp"
server_key: "2e1dbef8c27e8367bafcd0c9c74878f2"
server_key_iv: "cc854e1c1c8028dda75237fc1676afa7"
server_key_version: "encryptionkey@0"
I20260810 14:44:22.186059 26721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:44:22.189479 26742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:22.190827 26721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260810 14:44:22.190973 26721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "8f6e26ec58524e3f86bc564ddfd0e893"
format_stamp: "Formatted at 2026-08-10 14:44:22 on dist-test-slave-vkjp"
server_key: "2e1dbef8c27e8367bafcd0c9c74878f2"
server_key_iv: "cc854e1c1c8028dda75237fc1676afa7"
server_key_version: "encryptionkey@0"
I20260810 14:44:22.191329 26721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:22.213219 26721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:22.217474 26721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:22.217701 26721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:22.239609 26721 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:40341
I20260810 14:44:22.239643 26814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:40341 every 8 connection(s)
I20260810 14:44:22.241003 26721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:44:22.242588 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 26721
I20260810 14:44:22.242836 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:44:22.243693 25438 external_mini_cluster.cc:1428] Setting key 043794d2e854a94d90d6fae3ed6252d8
I20260810 14:44:22.245594 26815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:22.257998 26815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893: Bootstrap starting.
Aug 10 14:44:22 dist-test-slave-vkjp krb5kdc[26706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373061, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:22.262069 26815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:22.263502 26815 log.cc:824] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:22.266950 26815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893: No bootstrap required, opened a new log
I20260810 14:44:22.268296 26818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:22.245797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44072 (local address 127.24.215.190:40341)
0810 14:44:22.246559 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:22.246574 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:22.246613 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:44:22.248330 (+ 1717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:22.248350 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:22.248724 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:22.249123 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:22.250561 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:22.251963 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:22.253787 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:22.254871 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:22.255073 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:22.259474 (+ 4401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:22.259493 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:22.259515 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:22.259580 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:22.263675 (+ 4095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:22.264574 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:22.264580 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:22.264586 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:22.264758 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:22.265159 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:22.265164 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:22.265167 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:22.265848 (+ 681us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:22.266165 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:22.266470 (+ 305us) server_negotiation.cc:299] Negotiation successful
0810 14:44:22.266905 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":574,"thread_start_us":336,"threads_started":1}
I20260810 14:44:22.275173 26815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6e26ec58524e3f86bc564ddfd0e893" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40341 } }
I20260810 14:44:22.275532 26815 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:22.275632 26815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f6e26ec58524e3f86bc564ddfd0e893, State: Initialized, Role: FOLLOWER
I20260810 14:44:22.276329 26815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6e26ec58524e3f86bc564ddfd0e893" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40341 } }
I20260810 14:44:22.276616 26815 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:44:22.276752 26815 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:44:22.276932 26815 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:22.279063 26815 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6e26ec58524e3f86bc564ddfd0e893" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40341 } }
I20260810 14:44:22.279673 26815 leader_election.cc:302] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f6e26ec58524e3f86bc564ddfd0e893; no voters:
I20260810 14:44:22.280375 26815 leader_election.cc:288] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:44:22.280517 26821 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:22.281343 26821 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [term 1 LEADER]: Becoming Leader. State: Replica: 8f6e26ec58524e3f86bc564ddfd0e893, State: Running, Role: LEADER
I20260810 14:44:22.281942 26821 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6e26ec58524e3f86bc564ddfd0e893" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40341 } }
I20260810 14:44:22.283008 26815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:44:22.285241 26822 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f6e26ec58524e3f86bc564ddfd0e893" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6e26ec58524e3f86bc564ddfd0e893" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40341 } } }
I20260810 14:44:22.285641 26822 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:22.286089 26823 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f6e26ec58524e3f86bc564ddfd0e893. Latest consensus state: current_term: 1 leader_uuid: "8f6e26ec58524e3f86bc564ddfd0e893" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6e26ec58524e3f86bc564ddfd0e893" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40341 } } }
I20260810 14:44:22.286340 26823 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893 [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:22.288964 26827 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:44:22.294901 26827 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:44:22.303627 26827 catalog_manager.cc:1347] Generated new cluster ID: 1ada3b9b63dc4dd7a1703f6361d6ecba
I20260810 14:44:22.303756 26827 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:44:22.321602 26827 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:44:22.323048 26827 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:44:22.330229 26827 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893: Generated new TSK 0
I20260810 14:44:22.331038 26827 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:44:22.390982 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:40341
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:22.737660 26848 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:22.737944 26848 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:22.738104 26848 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:22.745528 26848 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:22.745637 26848 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:22.745798 26848 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:44:22.745931 26848 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:44:22.756498 26848 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.215.190:40341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:22.757889 26848 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:22.759402 26848 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:22.773211 26856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:22.773340 26853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:22.773236 26854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:22.774111 26848 server_base.cc:1034] running on GCE node
I20260810 14:44:22.774741 26848 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:22.775847 26848 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:22.777135 26848 hybrid_clock.cc:648] HybridClock initialized: now 1786373062777039 us; error 60 us; skew 500 ppm
I20260810 14:44:22.777561 26848 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:22 dist-test-slave-vkjp krb5kdc[26706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373062, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:22.785992 26848 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:44:22.787834 26848 webserver.cc:460] Webserver started at http://127.24.215.129:45771/ using document root <none> and password file <none>
I20260810 14:44:22.788681 26848 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:22.788827 26848 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:22.789240 26848 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:22.792523 26848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "722f1ef69fc84c7880bfca29f3910958"
format_stamp: "Formatted at 2026-08-10 14:44:22 on dist-test-slave-vkjp"
server_key: "376979fb846781be871dcee27b5bdf52"
server_key_iv: "5974926ed8106e06f41f2118d915a30c"
server_key_version: "encryptionkey@0"
I20260810 14:44:22.793430 26848 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "722f1ef69fc84c7880bfca29f3910958"
format_stamp: "Formatted at 2026-08-10 14:44:22 on dist-test-slave-vkjp"
server_key: "376979fb846781be871dcee27b5bdf52"
server_key_iv: "5974926ed8106e06f41f2118d915a30c"
server_key_version: "encryptionkey@0"
I20260810 14:44:22.799458 26848 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 14:44:22.803294 26866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:22.804716 26848 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:44:22.804921 26848 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "722f1ef69fc84c7880bfca29f3910958"
format_stamp: "Formatted at 2026-08-10 14:44:22 on dist-test-slave-vkjp"
server_key: "376979fb846781be871dcee27b5bdf52"
server_key_iv: "5974926ed8106e06f41f2118d915a30c"
server_key_version: "encryptionkey@0"
I20260810 14:44:22.805161 26848 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:22.832877 26848 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:22.837728 26848 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:22.838037 26848 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:22.841053 26848 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:22.841159 26848 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:22.841295 26848 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:22.841377 26848 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:44:22 dist-test-slave-vkjp krb5kdc[26706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373062, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:22.869712 26880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:22.842379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:40341 (local address 127.24.215.129:53287)
0810 14:44:22.843489 (+ 1110us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:22.843560 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:22.845850 (+ 2290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:22.847026 (+ 1176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:22.847046 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:22.847957 (+ 911us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:22.848904 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:22.848960 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:22.850728 (+ 1768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:22.850735 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:22.852855 (+ 2120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:22.852875 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:22.854144 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:22.854155 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:22.854456 (+ 301us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:22.855372 (+ 916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:22.855395 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:22.859114 (+ 3719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:22.863774 (+ 4660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:22.863784 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:22.863809 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:22.864729 (+ 920us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:22.865192 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:22.865204 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:22.865206 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:22.865576 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:22.866383 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:22.866389 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:22.866938 (+ 549us) client_negotiation.cc:769] Sending connection context
0810 14:44:22.867311 (+ 373us) client_negotiation.cc:240] Negotiation successful
0810 14:44:22.867939 (+ 628us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":705,"thread_start_us":415,"threads_started":1}
I20260810 14:44:22.870844 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:22.841710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:53287 (local address 127.24.215.190:40341)
0810 14:44:22.844222 (+ 2512us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:22.844229 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:22.844257 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:44:22.846225 (+ 1968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:22.846239 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:22.846357 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:22.846469 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:22.849191 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:22.850528 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:22.853042 (+ 2514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:22.853850 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:22.854020 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:22.859466 (+ 5446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:22.859477 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:22.859484 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:22.859532 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:22.863561 (+ 4029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:22.864901 (+ 1340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:22.864908 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:22.864923 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:22.865064 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:22.865754 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:22.865758 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:22.865760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:22.866145 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:22.866308 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:22.870319 (+ 4011us) server_negotiation.cc:299] Negotiation successful
0810 14:44:22.870520 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2364,"thread_start_us":2248,"threads_started":1}
I20260810 14:44:22.883936 26848 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:42511
I20260810 14:44:22.884039 27006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:42511 every 8 connection(s)
I20260810 14:44:22.885336 26848 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:44:22.887562 27007 heartbeater.cc:347] Connected to a master server at 127.24.215.190:40341
I20260810 14:44:22.887833 27007 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:22.888546 27007 heartbeater.cc:511] Master 127.24.215.190:40341 requested a full tablet report, sending...
I20260810 14:44:22.890990 26764 ts_manager.cc:194] Registered new tserver with Master: 722f1ef69fc84c7880bfca29f3910958 (127.24.215.129:42511)
I20260810 14:44:22.891978 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 26848
I20260810 14:44:22.892225 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:44:22.892871 25438 external_mini_cluster.cc:1428] Setting key 1d4353d1ae4dab94ad37e4c85171f578
I20260810 14:44:22.892908 26992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:22.878000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:40341 (local address 127.24.215.129:33323)
0810 14:44:22.878661 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:22.878678 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:22.878853 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:22.879290 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:22.879299 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:22.880081 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:22.880650 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:22.880665 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:22.881836 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:22.881841 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:22.883314 (+ 1473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:22.883339 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:22.884449 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:22.884454 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:22.884625 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:22.885271 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:22.885289 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:22.886183 (+ 894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:22.890027 (+ 3844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:22.890032 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:22.890036 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:22.890567 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:22.890923 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:22.890928 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:22.890931 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:22.891119 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:22.891743 (+ 624us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:22.891748 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:22.891965 (+ 217us) client_negotiation.cc:769] Sending connection context
0810 14:44:22.892136 (+ 171us) client_negotiation.cc:240] Negotiation successful
0810 14:44:22.892457 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":360,"threads_started":1}
I20260810 14:44:22.893014 26764 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:53287
I20260810 14:44:22.893958 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:22.877910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:33323 (local address 127.24.215.190:40341)
0810 14:44:22.878076 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:22.878080 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:22.878780 (+ 700us) server_negotiation.cc:407] Connection header received
0810 14:44:22.879017 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:22.879022 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:22.879147 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:22.879259 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:22.880819 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:22.881679 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:22.883517 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:22.884260 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:22.884400 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:22.886447 (+ 2047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:22.886454 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:22.886459 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:22.886500 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:22.889894 (+ 3394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:22.890675 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:22.890680 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:22.890682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:22.890810 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:22.891235 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:22.891242 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:22.891247 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:22.891595 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:22.891776 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:22.893341 (+ 1565us) server_negotiation.cc:299] Negotiation successful
0810 14:44:22.893584 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:44:22.939340 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:40341
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:23.269091 27011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:23.269376 27011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:23.269537 27011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:23.276954 27011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:23.277064 27011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:23.277202 27011 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:44:23.277334 27011 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:44:23.287403 27011 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.215.190:40341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:23.288800 27011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:23.290216 27011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:23.303738 27021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:23.303844 27019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:23.303821 27024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:23.304811 27011 server_base.cc:1034] running on GCE node
I20260810 14:44:23.305531 27011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:23.306465 27011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:23.307719 27011 hybrid_clock.cc:648] HybridClock initialized: now 1786373063307652 us; error 38 us; skew 500 ppm
I20260810 14:44:23.308037 27011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:23 dist-test-slave-vkjp krb5kdc[26706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373063, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:23.311667 27011 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:44:23.313179 27011 webserver.cc:460] Webserver started at http://127.24.215.130:46305/ using document root <none> and password file <none>
I20260810 14:44:23.313879 27011 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:23.314013 27011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:23.314363 27011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:23.317390 27011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "fae9e09726244718a664bda598cef4ea"
format_stamp: "Formatted at 2026-08-10 14:44:23 on dist-test-slave-vkjp"
server_key: "29eb29e614448c388d2f6de226ab8cf0"
server_key_iv: "723a9b1f87799be27c425bf98d20107a"
server_key_version: "encryptionkey@0"
I20260810 14:44:23.318236 27011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "fae9e09726244718a664bda598cef4ea"
format_stamp: "Formatted at 2026-08-10 14:44:23 on dist-test-slave-vkjp"
server_key: "29eb29e614448c388d2f6de226ab8cf0"
server_key_iv: "723a9b1f87799be27c425bf98d20107a"
server_key_version: "encryptionkey@0"
I20260810 14:44:23.323602 27011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 14:44:23.327214 27034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:23.328565 27011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:44:23.328753 27011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "fae9e09726244718a664bda598cef4ea"
format_stamp: "Formatted at 2026-08-10 14:44:23 on dist-test-slave-vkjp"
server_key: "29eb29e614448c388d2f6de226ab8cf0"
server_key_iv: "723a9b1f87799be27c425bf98d20107a"
server_key_version: "encryptionkey@0"
I20260810 14:44:23.329032 27011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:23.341884 27011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:23.345844 27011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:23.346127 27011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:23.348942 27011 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:23.349069 27011 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:23.349215 27011 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:23.349261 27011 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:44:23 dist-test-slave-vkjp krb5kdc[26706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373063, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:23.375370 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:23.349279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:33967 (local address 127.24.215.190:40341)
0810 14:44:23.349530 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:23.349541 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:23.351134 (+ 1593us) server_negotiation.cc:407] Connection header received
0810 14:44:23.354148 (+ 3014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:23.354155 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:23.355114 (+ 959us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:23.355247 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:23.357975 (+ 2728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.359505 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.361725 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.362562 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.362753 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:23.367001 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:23.367012 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:23.367020 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:23.367076 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:23.370603 (+ 3527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:23.371774 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:23.371782 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:23.371786 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:23.371971 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:23.372616 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:23.372622 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:23.372625 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:23.373061 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:23.373218 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:23.374486 (+ 1268us) server_negotiation.cc:299] Negotiation successful
0810 14:44:23.374821 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 14:44:23.376495 27043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:23.349918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:40341 (local address 127.24.215.130:33967)
0810 14:44:23.350924 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:23.350982 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:23.353773 (+ 2791us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:23.355544 (+ 1771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:23.355557 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:23.356584 (+ 1027us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:23.357639 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:23.357674 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.359682 (+ 2008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.359691 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:23.361540 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:23.361565 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.362818 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.362823 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:23.363002 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:23.363780 (+ 778us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:23.363819 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:23.366760 (+ 2941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:23.370747 (+ 3987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:23.370758 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:23.370775 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:23.371629 (+ 854us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:23.372147 (+ 518us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:23.372155 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:23.372159 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:23.372483 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:23.373318 (+ 835us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:23.373324 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:23.373864 (+ 540us) client_negotiation.cc:769] Sending connection context
0810 14:44:23.374320 (+ 456us) client_negotiation.cc:240] Negotiation successful
0810 14:44:23.374802 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":406,"threads_started":1}
I20260810 14:44:23.396770 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:23.382485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:43653 (local address 127.24.215.190:40341)
0810 14:44:23.382666 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:23.382674 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:23.383309 (+ 635us) server_negotiation.cc:407] Connection header received
0810 14:44:23.383515 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:23.383520 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:23.383604 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:23.383708 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:23.384707 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.385455 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.386759 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.387306 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.387445 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:23.390237 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:23.390245 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:23.390251 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:23.390303 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:23.393747 (+ 3444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:23.394414 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:23.394420 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:23.394423 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:23.394573 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:23.395074 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:23.395129 (+ 55us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:23.395133 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:23.395472 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:23.395634 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:23.396172 (+ 538us) server_negotiation.cc:299] Negotiation successful
0810 14:44:23.396393 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 14:44:23.396993 27121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:23.382533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:40341 (local address 127.24.215.130:43653)
0810 14:44:23.383192 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:23.383207 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:23.383371 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:23.383735 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:23.383740 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:23.384213 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:23.384569 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:23.384579 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.385601 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.385606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:23.386609 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:23.386620 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.387541 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.387552 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:23.387835 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:23.388720 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:23.388749 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:23.389986 (+ 1237us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:23.393878 (+ 3892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:23.393883 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:23.393887 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:23.394286 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:23.394738 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:23.394743 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:23.394747 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:23.394955 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:23.395607 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:23.395612 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:23.395873 (+ 261us) client_negotiation.cc:769] Sending connection context
0810 14:44:23.396040 (+ 167us) client_negotiation.cc:240] Negotiation successful
0810 14:44:23.396408 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":328,"threads_started":1}
I20260810 14:44:23.407408 27011 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:33359
I20260810 14:44:23.407429 27171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:33359 every 8 connection(s)
I20260810 14:44:23.408687 27011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:44:23.409782 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 27011
I20260810 14:44:23.410061 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:44:23.410663 25438 external_mini_cluster.cc:1428] Setting key 03c103cc3e6ea612a70547c80c81a6da
I20260810 14:44:23.410823 27172 heartbeater.cc:347] Connected to a master server at 127.24.215.190:40341
I20260810 14:44:23.411164 27172 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:23.411818 27172 heartbeater.cc:511] Master 127.24.215.190:40341 requested a full tablet report, sending...
I20260810 14:44:23.413281 26763 ts_manager.cc:194] Registered new tserver with Master: fae9e09726244718a664bda598cef4ea (127.24.215.130:33359)
I20260810 14:44:23.414638 26763 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:33967
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:44:23.453905 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:40341
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:39013
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:23.786844 27177 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:23.787237 27177 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:23.787410 27177 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:23.794981 27177 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:23.795154 27177 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:23.795310 27177 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:44:23.795463 27177 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:44:23.805502 27177 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:39013
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.215.190:40341
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:23.806875 27177 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:23.808395 27177 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:23.821485 27184 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:23.821445 27187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:23.821516 27183 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:23.822353 27177 server_base.cc:1034] running on GCE node
I20260810 14:44:23.822911 27177 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:23.823853 27177 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:23.825081 27177 hybrid_clock.cc:648] HybridClock initialized: now 1786373063825034 us; error 27 us; skew 500 ppm
I20260810 14:44:23.825413 27177 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:23 dist-test-slave-vkjp krb5kdc[26706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373063, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:23.828666 27177 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:44:23.830163 27177 webserver.cc:460] Webserver started at http://127.24.215.131:35467/ using document root <none> and password file <none>
I20260810 14:44:23.830869 27177 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:23.831002 27177 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:23.831351 27177 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:23.834363 27177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "d248832de2f24955bd5f623cc914eecd"
format_stamp: "Formatted at 2026-08-10 14:44:23 on dist-test-slave-vkjp"
server_key: "7281547bc1a80f41119f39b1825ba2d6"
server_key_iv: "273ece6beb67dd3c987b5555201e7739"
server_key_version: "encryptionkey@0"
I20260810 14:44:23.835275 27177 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "d248832de2f24955bd5f623cc914eecd"
format_stamp: "Formatted at 2026-08-10 14:44:23 on dist-test-slave-vkjp"
server_key: "7281547bc1a80f41119f39b1825ba2d6"
server_key_iv: "273ece6beb67dd3c987b5555201e7739"
server_key_version: "encryptionkey@0"
I20260810 14:44:23.840512 27177 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:44:23.843976 27194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:23.845263 27177 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:44:23.845445 27177 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "d248832de2f24955bd5f623cc914eecd"
format_stamp: "Formatted at 2026-08-10 14:44:23 on dist-test-slave-vkjp"
server_key: "7281547bc1a80f41119f39b1825ba2d6"
server_key_iv: "273ece6beb67dd3c987b5555201e7739"
server_key_version: "encryptionkey@0"
I20260810 14:44:23.845661 27177 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:23.862664 27177 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:23.866945 27177 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:23.867280 27177 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:23.870100 27177 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:23.870208 27177 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:23.870344 27177 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:23.870442 27177 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:44:23 dist-test-slave-vkjp krb5kdc[26706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373063, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:23.897881 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:23.870681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:39869 (local address 127.24.215.190:40341)
0810 14:44:23.870886 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:23.870893 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:23.872483 (+ 1590us) server_negotiation.cc:407] Connection header received
0810 14:44:23.875071 (+ 2588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:23.875100 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:23.875209 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:23.875327 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:23.878387 (+ 3060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.879793 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.883266 (+ 3473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.883903 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.884032 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:23.889240 (+ 5208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:23.889248 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:23.889252 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:23.889303 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:23.892912 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:23.893960 (+ 1048us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:23.893969 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:23.893974 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:23.894143 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:23.894741 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:23.894746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:23.894749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:23.895174 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:23.895366 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:23.897241 (+ 1875us) server_negotiation.cc:299] Negotiation successful
0810 14:44:23.897504 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 14:44:23.898262 27007 heartbeater.cc:503] Master 127.24.215.190:40341 was elected leader, sending a full tablet report...
I20260810 14:44:23.898653 27203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:23.871297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:40341 (local address 127.24.215.131:39869)
0810 14:44:23.872266 (+ 969us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:23.872331 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:23.874682 (+ 2351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:23.875655 (+ 973us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:23.875673 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:23.876665 (+ 992us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:23.878094 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:23.878128 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.880027 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.880034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:23.882999 (+ 2965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:23.883016 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.884370 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.884379 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:23.884752 (+ 373us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:23.885606 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:23.885641 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:23.888942 (+ 3301us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:23.893063 (+ 4121us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:23.893073 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:23.893088 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:23.893760 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:23.894293 (+ 533us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:23.894297 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:23.894299 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:23.894567 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:23.895467 (+ 900us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:23.895478 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:23.896055 (+ 577us) client_negotiation.cc:769] Sending connection context
0810 14:44:23.896443 (+ 388us) client_negotiation.cc:240] Negotiation successful
0810 14:44:23.897163 (+ 720us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":418,"threads_started":1}
I20260810 14:44:23.915370 27177 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:34351
I20260810 14:44:23.915392 27331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:34351 every 8 connection(s)
I20260810 14:44:23.916942 27177 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:44:23.919147 27332 heartbeater.cc:347] Connected to a master server at 127.24.215.190:40341
I20260810 14:44:23.919495 27332 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:23.920140 27332 heartbeater.cc:511] Master 127.24.215.190:40341 requested a full tablet report, sending...
I20260810 14:44:23.921574 26763 ts_manager.cc:194] Registered new tserver with Master: d248832de2f24955bd5f623cc914eecd (127.24.215.131:34351)
I20260810 14:44:23.922475 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:23.906639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:51391 (local address 127.24.215.190:40341)
0810 14:44:23.907135 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:23.907142 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:23.907475 (+ 333us) server_negotiation.cc:407] Connection header received
0810 14:44:23.907684 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:23.907692 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:23.907782 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:23.907883 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:23.909450 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.910661 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.913319 (+ 2658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.913726 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.913906 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:23.916504 (+ 2598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:23.916510 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:23.916513 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:23.916545 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:23.919896 (+ 3351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:23.920592 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:23.920597 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:23.920601 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:23.920751 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:23.921184 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:23.921188 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:23.921190 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:23.921486 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:23.921604 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:23.922046 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 14:44:23.922221 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":315}
I20260810 14:44:23.922789 27305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:23.906728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:40341 (local address 127.24.215.131:51391)
0810 14:44:23.907373 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:23.907402 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:23.907550 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:23.908095 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:23.908102 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:23.908823 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:23.909298 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:23.909310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.910858 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.910868 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:23.913181 (+ 2313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:23.913196 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.914002 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.914008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:23.914230 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:23.915210 (+ 980us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:23.915237 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:23.916318 (+ 1081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:23.920024 (+ 3706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:23.920030 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:23.920035 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:23.920471 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:23.920867 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:23.920873 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:23.920876 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:23.921056 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:23.921615 (+ 559us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:23.921620 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:23.921807 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 14:44:23.921951 (+ 144us) client_negotiation.cc:240] Negotiation successful
0810 14:44:23.922257 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":423,"threads_started":1}
I20260810 14:44:23.922744 26763 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:39869
I20260810 14:44:23.923457 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 27177
I20260810 14:44:23.923648 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:44:23.924072 25438 external_mini_cluster.cc:1428] Setting key 58ab7e51eb82256b3bb5139ba87188fc
I20260810 14:44:23.927326 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:44:23 dist-test-slave-vkjp krb5kdc[26706](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373063, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 14:44:23 dist-test-slave-vkjp krb5kdc[26706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373063, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:23.977558 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:23.963061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44086 (local address 127.24.215.190:40341)
0810 14:44:23.963351 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:23.963356 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:23.963691 (+ 335us) server_negotiation.cc:407] Connection header received
0810 14:44:23.963836 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:23.963859 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:23.963949 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:23.964073 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:23.965467 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.966199 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.967607 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:23.968033 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:23.968131 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:23.972182 (+ 4051us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:23.972188 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:23.972192 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:23.972226 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:23.974955 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:23.975614 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:23.975618 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:23.975621 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:23.975744 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:23.976140 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:23.976145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:23.976147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:23.976412 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:23.976683 (+ 271us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:23.976973 (+ 290us) server_negotiation.cc:299] Negotiation successful
0810 14:44:23.977212 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129}
I20260810 14:44:23.982054 26764 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:44:23.989742 26764 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:44:24.017488 27353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.004860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34952 (local address 127.24.215.129:42511)
0810 14:44:24.007596 (+ 2736us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:24.007607 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:24.008319 (+ 712us) server_negotiation.cc:407] Connection header received
0810 14:44:24.008435 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:24.008443 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:24.008716 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:24.008937 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:24.009842 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.011225 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.013773 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.015231 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.016386 (+ 1155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.016576 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:24.016982 (+ 406us) server_negotiation.cc:299] Negotiation successful
0810 14:44:24.017156 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2561,"thread_start_us":2439,"threads_started":1}
I20260810 14:44:24.017511 27350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.004355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:42511 (local address 127.0.0.1:34952)
0810 14:44:24.007872 (+ 3517us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:24.007886 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:24.008025 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:24.009213 (+ 1188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:24.009221 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:24.009256 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:24.009623 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.009637 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.011381 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.011386 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.013518 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.013530 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.016390 (+ 2860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.016397 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.016740 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.016818 (+ 78us) client_negotiation.cc:769] Sending connection context
0810 14:44:24.016920 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 14:44:24.017137 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3420,"thread_start_us":655,"threads_started":1}
I20260810 14:44:24.018096 27348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.003196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:33359 (local address 127.0.0.1:33208)
0810 14:44:24.003998 (+ 802us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:24.004034 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:24.004216 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:24.010451 (+ 6235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:24.010455 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:24.010481 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:24.010959 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.010969 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.012974 (+ 2005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.012985 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.015349 (+ 2364us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.015359 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.017112 (+ 1753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.017118 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.017297 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.017339 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 14:44:24.017426 (+ 87us) client_negotiation.cc:240] Negotiation successful
0810 14:44:24.017726 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":539,"mutex_wait_us":21,"thread_start_us":385,"threads_started":1}
I20260810 14:44:24.018777 27349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.003368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33208 (local address 127.24.215.130:33359)
0810 14:44:24.009885 (+ 6517us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:24.009893 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:24.009924 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:44:24.010024 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:24.010029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:24.010218 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:24.011186 (+ 968us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:24.011389 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.012814 (+ 1425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.015532 (+ 2718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.016896 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.017070 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.017219 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:24.018353 (+ 1134us) server_negotiation.cc:299] Negotiation successful
0810 14:44:24.018534 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6330,"thread_start_us":6195,"threads_started":1}
I20260810 14:44:24.020013 27351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.003748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:34351 (local address 127.0.0.1:45862)
0810 14:44:24.004881 (+ 1133us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:24.004898 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:24.005016 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:24.010253 (+ 5237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:24.010259 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:24.010285 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:24.010781 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.010796 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.012766 (+ 1970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.012774 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.015328 (+ 2554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.015344 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.019236 (+ 3892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.019244 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.019409 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.019440 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 14:44:24.019534 (+ 94us) client_negotiation.cc:240] Negotiation successful
0810 14:44:24.019668 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":993,"mutex_wait_us":95,"thread_start_us":3902,"threads_started":1}
I20260810 14:44:24.020074 27352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.004510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45862 (local address 127.24.215.131:34351)
0810 14:44:24.009596 (+ 5086us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:24.009606 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:24.009638 (+ 32us) server_negotiation.cc:407] Connection header received
0810 14:44:24.009737 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:24.009748 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:24.010047 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:24.010242 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:24.011028 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.012630 (+ 1602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.017312 (+ 4682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.018711 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.018920 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.019075 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:24.019603 (+ 528us) server_negotiation.cc:299] Negotiation successful
0810 14:44:24.019775 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4895,"thread_start_us":4641,"threads_started":1}
I20260810 14:44:24.021473 26927 tablet_service.cc:1467] Processing CreateTablet for tablet c8c2b73200e44767a129804987ca2e3c (DEFAULT_TABLE table=test-table [id=c116d4bb4bb444d3a303ad13f317cdf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:24.022821 26927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8c2b73200e44767a129804987ca2e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:24.023192 27251 tablet_service.cc:1467] Processing CreateTablet for tablet c8c2b73200e44767a129804987ca2e3c (DEFAULT_TABLE table=test-table [id=c116d4bb4bb444d3a303ad13f317cdf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:24.024632 27251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8c2b73200e44767a129804987ca2e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:24.029640 27088 tablet_service.cc:1467] Processing CreateTablet for tablet c8c2b73200e44767a129804987ca2e3c (DEFAULT_TABLE table=test-table [id=c116d4bb4bb444d3a303ad13f317cdf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:24.031020 27088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c8c2b73200e44767a129804987ca2e3c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:24.034565 27355 tablet_bootstrap.cc:492] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958: Bootstrap starting.
I20260810 14:44:24.038090 27355 tablet_bootstrap.cc:654] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:24.038539 27356 tablet_bootstrap.cc:492] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd: Bootstrap starting.
I20260810 14:44:24.039613 27355 log.cc:824] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:24.042003 27356 tablet_bootstrap.cc:654] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:24.042809 27355 tablet_bootstrap.cc:492] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958: No bootstrap required, opened a new log
I20260810 14:44:24.043124 27355 ts_tablet_manager.cc:1397] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260810 14:44:24.043275 27356 log.cc:824] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:24.045369 27359 tablet_bootstrap.cc:492] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea: Bootstrap starting.
I20260810 14:44:24.047379 27356 tablet_bootstrap.cc:492] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd: No bootstrap required, opened a new log
I20260810 14:44:24.047654 27356 ts_tablet_manager.cc:1397] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260810 14:44:24.048653 27359 tablet_bootstrap.cc:654] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:24.049777 27359 log.cc:824] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:24.052511 27359 tablet_bootstrap.cc:492] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea: No bootstrap required, opened a new log
I20260810 14:44:24.052848 27359 ts_tablet_manager.cc:1397] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 14:44:24.053933 27355 raft_consensus.cc:348] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.054491 27355 raft_consensus.cc:374] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:24.054605 27355 raft_consensus.cc:729] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 722f1ef69fc84c7880bfca29f3910958, State: Initialized, Role: FOLLOWER
I20260810 14:44:24.055301 27355 consensus_queue.cc:260] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.056998 27355 ts_tablet_manager.cc:1428] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958: Time spent starting tablet: real 0.014s user 0.011s sys 0.002s
I20260810 14:44:24.058399 27356 raft_consensus.cc:348] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.058845 27356 raft_consensus.cc:374] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:24.058971 27356 raft_consensus.cc:729] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d248832de2f24955bd5f623cc914eecd, State: Initialized, Role: FOLLOWER
I20260810 14:44:24.059604 27356 consensus_queue.cc:260] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.060845 27332 heartbeater.cc:503] Master 127.24.215.190:40341 was elected leader, sending a full tablet report...
I20260810 14:44:24.061573 27356 ts_tablet_manager.cc:1428] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260810 14:44:24.062602 27359 raft_consensus.cc:348] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.063118 27359 raft_consensus.cc:374] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:24.063241 27359 raft_consensus.cc:729] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae9e09726244718a664bda598cef4ea, State: Initialized, Role: FOLLOWER
I20260810 14:44:24.063973 27359 consensus_queue.cc:260] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.065106 27172 heartbeater.cc:503] Master 127.24.215.190:40341 was elected leader, sending a full tablet report...
I20260810 14:44:24.065826 27359 ts_tablet_manager.cc:1428] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea: Time spent starting tablet: real 0.013s user 0.006s sys 0.008s
I20260810 14:44:24.114203 27363 raft_consensus.cc:482] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:44:24.114826 27363 raft_consensus.cc:504] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.116618 27363 leader_election.cc:288] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 722f1ef69fc84c7880bfca29f3910958 (127.24.215.129:42511), d248832de2f24955bd5f623cc914eecd (127.24.215.131:34351)
I20260810 14:44:24.125934 27353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.117098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:51183 (local address 127.24.215.129:42511)
0810 14:44:24.117448 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:24.117458 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:24.117834 (+ 376us) server_negotiation.cc:407] Connection header received
0810 14:44:24.118100 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:24.118104 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:24.118213 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:24.118319 (+ 106us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:24.119197 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.120347 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.123585 (+ 3238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.125045 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.125216 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.125330 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:24.125607 (+ 277us) server_negotiation.cc:299] Negotiation successful
0810 14:44:24.125726 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260810 14:44:24.126374 27352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.117206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:55025 (local address 127.24.215.131:34351)
0810 14:44:24.117442 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:24.117452 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:24.118049 (+ 597us) server_negotiation.cc:407] Connection header received
0810 14:44:24.118193 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:24.118199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:24.118276 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:24.118364 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:24.119261 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.120347 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.123625 (+ 3278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.125387 (+ 1762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.125551 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.125621 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:24.125964 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 14:44:24.126117 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 14:44:24.126423 27364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.116976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:34351 (local address 127.24.215.130:55025)
0810 14:44:24.117938 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:24.117955 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:24.118073 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:24.118405 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:24.118408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:24.118424 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:24.119057 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.119070 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.120506 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.120511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.123401 (+ 2890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.123411 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.125605 (+ 2194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.125611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.125785 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.125818 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:44:24.125917 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 14:44:24.126040 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":846,"mutex_wait_us":305,"thread_start_us":454,"threads_started":1}
I20260810 14:44:24.126780 27365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.116976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:42511 (local address 127.24.215.130:51183)
0810 14:44:24.117697 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:24.117720 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:24.117908 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:24.118386 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:24.118390 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:24.118417 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:24.118993 (+ 576us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.119011 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.120506 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.120511 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.123382 (+ 2871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.123404 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.125276 (+ 1872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.125281 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.125404 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.125431 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 14:44:24.125567 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 14:44:24.125721 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"mutex_wait_us":206,"thread_start_us":629,"threads_started":1}
I20260810 14:44:24.127737 26949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8c2b73200e44767a129804987ca2e3c" candidate_uuid: "fae9e09726244718a664bda598cef4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "722f1ef69fc84c7880bfca29f3910958" is_pre_election: true
I20260810 14:44:24.127738 27273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8c2b73200e44767a129804987ca2e3c" candidate_uuid: "fae9e09726244718a664bda598cef4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d248832de2f24955bd5f623cc914eecd" is_pre_election: true
I20260810 14:44:24.128253 27273 raft_consensus.cc:2454] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fae9e09726244718a664bda598cef4ea in term 0.
I20260810 14:44:24.128252 26949 raft_consensus.cc:2454] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fae9e09726244718a664bda598cef4ea in term 0.
I20260810 14:44:24.129034 27037 leader_election.cc:302] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 722f1ef69fc84c7880bfca29f3910958, fae9e09726244718a664bda598cef4ea; no voters:
I20260810 14:44:24.129791 27363 raft_consensus.cc:2781] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:44:24.129988 27363 raft_consensus.cc:482] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:44:24.130116 27363 raft_consensus.cc:3037] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:24.132607 27363 raft_consensus.cc:504] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.133399 27363 leader_election.cc:288] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [CANDIDATE]: Term 1 election: Requested vote from peers 722f1ef69fc84c7880bfca29f3910958 (127.24.215.129:42511), d248832de2f24955bd5f623cc914eecd (127.24.215.131:34351)
I20260810 14:44:24.133790 26949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8c2b73200e44767a129804987ca2e3c" candidate_uuid: "fae9e09726244718a664bda598cef4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "722f1ef69fc84c7880bfca29f3910958"
I20260810 14:44:24.134004 26949 raft_consensus.cc:3037] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:24.133999 27273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c8c2b73200e44767a129804987ca2e3c" candidate_uuid: "fae9e09726244718a664bda598cef4ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d248832de2f24955bd5f623cc914eecd"
I20260810 14:44:24.134241 27273 raft_consensus.cc:3037] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:24.136500 26949 raft_consensus.cc:2454] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae9e09726244718a664bda598cef4ea in term 1.
I20260810 14:44:24.136672 27273 raft_consensus.cc:2454] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fae9e09726244718a664bda598cef4ea in term 1.
I20260810 14:44:24.136933 27037 leader_election.cc:302] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 722f1ef69fc84c7880bfca29f3910958, fae9e09726244718a664bda598cef4ea; no voters:
I20260810 14:44:24.137290 27363 raft_consensus.cc:2781] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:24.137847 27363 raft_consensus.cc:686] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [term 1 LEADER]: Becoming Leader. State: Replica: fae9e09726244718a664bda598cef4ea, State: Running, Role: LEADER
W20260810 14:44:24.137920 27008 tablet.cc:2369] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:44:24.138411 27363 consensus_queue.cc:237] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.142937 26763 catalog_manager.cc:5495] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea reported cstate change: term changed from 0 to 1, leader changed from <none> to fae9e09726244718a664bda598cef4ea (127.24.215.130). New cstate: current_term: 1 leader_uuid: "fae9e09726244718a664bda598cef4ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } health_report { overall_health: UNKNOWN } } }
W20260810 14:44:24.161476 27173 tablet.cc:2369] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:44:24.169273 27333 tablet.cc:2369] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:44:24.179263 27349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.173031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33214 (local address 127.24.215.130:33359)
0810 14:44:24.173268 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:24.173328 (+ 60us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:24.173360 (+ 32us) server_negotiation.cc:407] Connection header received
0810 14:44:24.173742 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:24.173746 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:24.173847 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:24.174003 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:24.174427 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.175253 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.176698 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.177164 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.177317 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.177876 (+ 559us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:24.178446 (+ 570us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:24.178648 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:24.178849 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 14:44:24.178939 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 14:44:24.187709 27273 raft_consensus.cc:1261] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd [term 1 FOLLOWER]: Refusing update from remote peer fae9e09726244718a664bda598cef4ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:24.187709 26949 raft_consensus.cc:1261] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958 [term 1 FOLLOWER]: Refusing update from remote peer fae9e09726244718a664bda598cef4ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:24.188724 27366 consensus_queue.cc:1035] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:24.189126 27363 consensus_queue.cc:1035] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:24.221010 26763 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:44:24.221442 26763 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:44:24.225809 26763 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893: Sending DeleteTablet for 3 replicas of tablet c8c2b73200e44767a129804987ca2e3c
I20260810 14:44:24.227180 27088 tablet_service.cc:1514] Processing DeleteTablet for tablet c8c2b73200e44767a129804987ca2e3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:24 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:33208
I20260810 14:44:24.227252 27251 tablet_service.cc:1514] Processing DeleteTablet for tablet c8c2b73200e44767a129804987ca2e3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:24 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:45862
I20260810 14:44:24.227411 26927 tablet_service.cc:1514] Processing DeleteTablet for tablet c8c2b73200e44767a129804987ca2e3c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:24 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:34952
I20260810 14:44:24.229468 26763 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44086:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:44:24.230306 26763 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:44:24.232788 27382 tablet_replica.cc:330] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958: stopping tablet replica
I20260810 14:44:24.233062 27380 tablet_replica.cc:330] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd: stopping tablet replica
I20260810 14:44:24.233775 27382 raft_consensus.cc:2229] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:44:24.234229 27380 raft_consensus.cc:2229] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:44:24.234748 27380 raft_consensus.cc:2258] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:24.238391 27380 ts_tablet_manager.cc:1905] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:24.246264 27251 tablet_service.cc:1467] Processing CreateTablet for tablet 13b38983588a4df5a0eacb18d589b399 (DEFAULT_TABLE table=test-table [id=a757cf518b794f7e8015fb6a5eaa5546]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:24.246773 27251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13b38983588a4df5a0eacb18d589b399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:24.247067 26927 tablet_service.cc:1467] Processing CreateTablet for tablet 13b38983588a4df5a0eacb18d589b399 (DEFAULT_TABLE table=test-table [id=a757cf518b794f7e8015fb6a5eaa5546]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:24.247640 26927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13b38983588a4df5a0eacb18d589b399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:24.249574 27087 tablet_service.cc:1467] Processing CreateTablet for tablet 13b38983588a4df5a0eacb18d589b399 (DEFAULT_TABLE table=test-table [id=a757cf518b794f7e8015fb6a5eaa5546]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:24.250097 27380 ts_tablet_manager.cc:1918] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:44:24.250233 27087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 13b38983588a4df5a0eacb18d589b399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:24.250645 27380 log.cc:1196] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/c8c2b73200e44767a129804987ca2e3c
I20260810 14:44:24.251545 27380 ts_tablet_manager.cc:1939] T c8c2b73200e44767a129804987ca2e3c P d248832de2f24955bd5f623cc914eecd: Deleting consensus metadata
I20260810 14:44:24.254251 26744 catalog_manager.cc:4841] TS d248832de2f24955bd5f623cc914eecd (127.24.215.131:34351): tablet c8c2b73200e44767a129804987ca2e3c (table test-table [id=c116d4bb4bb444d3a303ad13f317cdf2]) successfully deleted
I20260810 14:44:24.257427 27359 tablet_bootstrap.cc:492] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea: Bootstrap starting.
I20260810 14:44:24.257588 27355 tablet_bootstrap.cc:492] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958: Bootstrap starting.
I20260810 14:44:24.259395 27359 tablet_bootstrap.cc:654] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:24.261379 27355 tablet_bootstrap.cc:654] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:24.262581 27356 tablet_bootstrap.cc:492] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd: Bootstrap starting.
I20260810 14:44:24.265035 27356 tablet_bootstrap.cc:654] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:24.269817 27382 raft_consensus.cc:2258] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:24.272336 27382 ts_tablet_manager.cc:1905] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:24.272668 27381 tablet_replica.cc:330] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea: stopping tablet replica
I20260810 14:44:24.273181 27381 raft_consensus.cc:2229] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:44:24.273833 27381 raft_consensus.cc:2258] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:24.275553 27355 tablet_bootstrap.cc:492] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958: No bootstrap required, opened a new log
I20260810 14:44:24.275746 27355 ts_tablet_manager.cc:1397] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.009s
I20260810 14:44:24.276382 27359 tablet_bootstrap.cc:492] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea: No bootstrap required, opened a new log
I20260810 14:44:24.276527 27359 ts_tablet_manager.cc:1397] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.005s
I20260810 14:44:24.277575 27356 tablet_bootstrap.cc:492] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd: No bootstrap required, opened a new log
I20260810 14:44:24.277765 27356 ts_tablet_manager.cc:1397] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd: Time spent bootstrapping tablet: real 0.015s user 0.003s sys 0.007s
I20260810 14:44:24.278208 27359 raft_consensus.cc:348] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.278425 27359 raft_consensus.cc:374] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:24.278517 27359 raft_consensus.cc:729] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae9e09726244718a664bda598cef4ea, State: Initialized, Role: FOLLOWER
I20260810 14:44:24.278793 27356 raft_consensus.cc:348] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.279067 27356 raft_consensus.cc:374] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:24.279207 27356 raft_consensus.cc:729] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d248832de2f24955bd5f623cc914eecd, State: Initialized, Role: FOLLOWER
I20260810 14:44:24.279322 27381 ts_tablet_manager.cc:1905] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:24.279582 27356 consensus_queue.cc:260] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.283151 27356 ts_tablet_manager.cc:1428] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd: Time spent starting tablet: real 0.005s user 0.000s sys 0.002s
I20260810 14:44:24.283401 27359 consensus_queue.cc:260] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.285899 27355 raft_consensus.cc:348] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.286201 27355 raft_consensus.cc:374] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:24.286252 27359 ts_tablet_manager.cc:1428] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea: Time spent starting tablet: real 0.010s user 0.002s sys 0.001s
I20260810 14:44:24.286324 27355 raft_consensus.cc:729] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 722f1ef69fc84c7880bfca29f3910958, State: Initialized, Role: FOLLOWER
I20260810 14:44:24.286700 27355 consensus_queue.cc:260] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.287580 27355 ts_tablet_manager.cc:1428] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958: Time spent starting tablet: real 0.012s user 0.001s sys 0.001s
I20260810 14:44:24.288306 27381 ts_tablet_manager.cc:1918] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:44:24.288535 27381 log.cc:1196] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/c8c2b73200e44767a129804987ca2e3c
I20260810 14:44:24.289238 27381 ts_tablet_manager.cc:1939] T c8c2b73200e44767a129804987ca2e3c P fae9e09726244718a664bda598cef4ea: Deleting consensus metadata
I20260810 14:44:24.291342 26745 catalog_manager.cc:4841] TS fae9e09726244718a664bda598cef4ea (127.24.215.130:33359): tablet c8c2b73200e44767a129804987ca2e3c (table test-table [id=c116d4bb4bb444d3a303ad13f317cdf2]) successfully deleted
I20260810 14:44:24.293330 27382 ts_tablet_manager.cc:1918] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:44:24.293501 27382 log.cc:1196] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/c8c2b73200e44767a129804987ca2e3c
I20260810 14:44:24.294082 27382 ts_tablet_manager.cc:1939] T c8c2b73200e44767a129804987ca2e3c P 722f1ef69fc84c7880bfca29f3910958: Deleting consensus metadata
I20260810 14:44:24.294225 27362 raft_consensus.cc:482] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:44:24.294471 27362 raft_consensus.cc:504] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.295879 26745 catalog_manager.cc:4841] TS 722f1ef69fc84c7880bfca29f3910958 (127.24.215.129:42511): tablet c8c2b73200e44767a129804987ca2e3c (table test-table [id=c116d4bb4bb444d3a303ad13f317cdf2]) successfully deleted
I20260810 14:44:24.296542 27362 leader_election.cc:288] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 722f1ef69fc84c7880bfca29f3910958 (127.24.215.129:42511), fae9e09726244718a664bda598cef4ea (127.24.215.130:33359)
I20260810 14:44:24.304665 27203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.296697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:42511 (local address 127.24.215.131:38317)
0810 14:44:24.296881 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:24.296901 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:24.297016 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:24.298069 (+ 1053us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:24.298074 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:24.298109 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:24.298355 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.298369 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.299661 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.299669 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.301870 (+ 2201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.301886 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.303795 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.303800 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.304043 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.304081 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 14:44:24.304186 (+ 105us) client_negotiation.cc:240] Negotiation successful
0810 14:44:24.304328 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":94}
I20260810 14:44:24.304714 27353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.297451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:38317 (local address 127.24.215.129:42511)
0810 14:44:24.297642 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:24.297648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:24.297671 (+ 23us) server_negotiation.cc:407] Connection header received
0810 14:44:24.297741 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:24.297746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:24.297856 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:24.297965 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:24.298508 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.299532 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.302061 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.303434 (+ 1373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.303632 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.303708 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:24.304250 (+ 542us) server_negotiation.cc:299] Negotiation successful
0810 14:44:24.304399 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 14:44:24.305876 27386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.297056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:33359 (local address 127.24.215.131:56811)
0810 14:44:24.298105 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:24.298122 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:24.298244 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:24.298641 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:24.298648 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:24.298674 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:24.299028 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.299039 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.300533 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.300538 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.302884 (+ 2346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.302898 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.304951 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.304957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.305130 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.305166 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 14:44:24.305327 (+ 161us) client_negotiation.cc:240] Negotiation successful
0810 14:44:24.305465 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":937,"mutex_wait_us":1,"thread_start_us":745,"threads_started":1}
I20260810 14:44:24.305912 27349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.297213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:56811 (local address 127.24.215.130:33359)
0810 14:44:24.297411 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:24.297419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:24.298224 (+ 805us) server_negotiation.cc:407] Connection header received
0810 14:44:24.298408 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:24.298414 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:24.298504 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:24.298608 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:24.299217 (+ 609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.300373 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.303064 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.304539 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.304660 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.304749 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:24.305354 (+ 605us) server_negotiation.cc:299] Negotiation successful
0810 14:44:24.305494 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 14:44:24.306002 26949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13b38983588a4df5a0eacb18d589b399" candidate_uuid: "d248832de2f24955bd5f623cc914eecd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "722f1ef69fc84c7880bfca29f3910958" is_pre_election: true
I20260810 14:44:24.306340 26949 raft_consensus.cc:2454] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d248832de2f24955bd5f623cc914eecd in term 0.
I20260810 14:44:24.306892 27197 leader_election.cc:302] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 722f1ef69fc84c7880bfca29f3910958, d248832de2f24955bd5f623cc914eecd; no voters:
I20260810 14:44:24.306941 27109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13b38983588a4df5a0eacb18d589b399" candidate_uuid: "d248832de2f24955bd5f623cc914eecd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae9e09726244718a664bda598cef4ea" is_pre_election: true
I20260810 14:44:24.307431 27362 raft_consensus.cc:2781] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:44:24.307566 27362 raft_consensus.cc:482] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:44:24.307672 27362 raft_consensus.cc:3037] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:24.307555 27109 raft_consensus.cc:2454] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d248832de2f24955bd5f623cc914eecd in term 0.
I20260810 14:44:24.310981 27362 raft_consensus.cc:504] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.311848 27362 leader_election.cc:288] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [CANDIDATE]: Term 1 election: Requested vote from peers 722f1ef69fc84c7880bfca29f3910958 (127.24.215.129:42511), fae9e09726244718a664bda598cef4ea (127.24.215.130:33359)
I20260810 14:44:24.312348 26949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13b38983588a4df5a0eacb18d589b399" candidate_uuid: "d248832de2f24955bd5f623cc914eecd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "722f1ef69fc84c7880bfca29f3910958"
I20260810 14:44:24.312459 27109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "13b38983588a4df5a0eacb18d589b399" candidate_uuid: "d248832de2f24955bd5f623cc914eecd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae9e09726244718a664bda598cef4ea"
I20260810 14:44:24.312675 27109 raft_consensus.cc:3037] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:24.312682 26949 raft_consensus.cc:3037] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:24.315886 26949 raft_consensus.cc:2454] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d248832de2f24955bd5f623cc914eecd in term 1.
I20260810 14:44:24.316318 27197 leader_election.cc:302] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 722f1ef69fc84c7880bfca29f3910958, d248832de2f24955bd5f623cc914eecd; no voters:
I20260810 14:44:24.316290 27109 raft_consensus.cc:2454] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d248832de2f24955bd5f623cc914eecd in term 1.
I20260810 14:44:24.316639 27362 raft_consensus.cc:2781] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:24.317200 27362 raft_consensus.cc:686] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [term 1 LEADER]: Becoming Leader. State: Replica: d248832de2f24955bd5f623cc914eecd, State: Running, Role: LEADER
I20260810 14:44:24.317613 27362 consensus_queue.cc:237] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.321362 26764 catalog_manager.cc:5495] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd reported cstate change: term changed from 0 to 1, leader changed from <none> to d248832de2f24955bd5f623cc914eecd (127.24.215.131). New cstate: current_term: 1 leader_uuid: "d248832de2f24955bd5f623cc914eecd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } health_report { overall_health: HEALTHY } } }
Aug 10 14:44:24 dist-test-slave-vkjp krb5kdc[26706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373062, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:24.382941 26878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.366183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44102 (local address 127.24.215.190:40341)
0810 14:44:24.366415 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:24.366422 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:24.367056 (+ 634us) server_negotiation.cc:407] Connection header received
0810 14:44:24.367273 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:24.367278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:24.367393 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:24.367513 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:24.368806 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.369982 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.371416 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.372119 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.372256 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.376205 (+ 3949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:24.376211 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:24.376215 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:24.376247 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:24.379493 (+ 3246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:24.380420 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:24.380425 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:24.380427 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:24.380622 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:24.381111 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:24.381116 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:24.381118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:24.381527 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:24.381680 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:24.382198 (+ 518us) server_negotiation.cc:299] Negotiation successful
0810 14:44:24.382576 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":166,"server-negotiator.queue_time_us":70}
I20260810 14:44:24.382979 27396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.366253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:40341 (local address 127.0.0.1:44102)
0810 14:44:24.366943 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:24.366960 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:24.367137 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:24.367571 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:24.367575 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:24.368335 (+ 760us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:24.368643 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.368652 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.370125 (+ 1473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.370129 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.371281 (+ 1152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.371293 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.372381 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.372386 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.372506 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.373251 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:24.373268 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:24.375956 (+ 2688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:24.379632 (+ 3676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:24.379647 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:24.379656 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:24.380254 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:24.380742 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:24.380747 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:24.380751 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:24.380931 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:24.381690 (+ 759us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:24.381700 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:24.381912 (+ 212us) client_negotiation.cc:769] Sending connection context
0810 14:44:24.382096 (+ 184us) client_negotiation.cc:240] Negotiation successful
0810 14:44:24.382423 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":382,"threads_started":1}
I20260810 14:44:24.389206 26764 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:44102:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 14:44:24.390393 26764 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:44:24.399240 27251 tablet_service.cc:1467] Processing CreateTablet for tablet c36ce25c1d4b4ccca5599ce7b426ff1e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2a59e4efdc8c4d6a8c9124ec3bffc353]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:44:24.399588 27088 tablet_service.cc:1467] Processing CreateTablet for tablet c36ce25c1d4b4ccca5599ce7b426ff1e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2a59e4efdc8c4d6a8c9124ec3bffc353]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:44:24.399861 27251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36ce25c1d4b4ccca5599ce7b426ff1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:24.400267 27088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36ce25c1d4b4ccca5599ce7b426ff1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:24.401711 26927 tablet_service.cc:1467] Processing CreateTablet for tablet c36ce25c1d4b4ccca5599ce7b426ff1e (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=2a59e4efdc8c4d6a8c9124ec3bffc353]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:44:24.402328 26927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c36ce25c1d4b4ccca5599ce7b426ff1e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:24.408084 27356 tablet_bootstrap.cc:492] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd: Bootstrap starting.
I20260810 14:44:24.409847 27356 tablet_bootstrap.cc:654] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:24.412277 27356 tablet_bootstrap.cc:492] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd: No bootstrap required, opened a new log
I20260810 14:44:24.412483 27356 ts_tablet_manager.cc:1397] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260810 14:44:24.413210 27356 raft_consensus.cc:348] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.413403 27356 raft_consensus.cc:374] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:24.413501 27356 raft_consensus.cc:729] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d248832de2f24955bd5f623cc914eecd, State: Initialized, Role: FOLLOWER
I20260810 14:44:24.413755 27356 consensus_queue.cc:260] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.414268 27355 tablet_bootstrap.cc:492] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958: Bootstrap starting.
I20260810 14:44:24.414358 27356 ts_tablet_manager.cc:1428] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 14:44:24.414605 27362 tablet_replica.cc:439] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } } }
I20260810 14:44:24.414865 27362 tablet_replica.cc:442] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:44:24.416216 27355 tablet_bootstrap.cc:654] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:24.418865 27355 tablet_bootstrap.cc:492] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958: No bootstrap required, opened a new log
I20260810 14:44:24.419034 27355 ts_tablet_manager.cc:1397] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 14:44:24.419126 27359 tablet_bootstrap.cc:492] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea: Bootstrap starting.
I20260810 14:44:24.419770 27355 raft_consensus.cc:348] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.420141 27355 raft_consensus.cc:374] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:24.420238 27355 raft_consensus.cc:729] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 722f1ef69fc84c7880bfca29f3910958, State: Initialized, Role: FOLLOWER
I20260810 14:44:24.420650 27355 consensus_queue.cc:260] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.421428 27359 tablet_bootstrap.cc:654] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:24.421176 27361 tablet_replica.cc:439] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } } }
I20260810 14:44:24.421566 27361 tablet_replica.cc:442] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:44:24.423053 27355 ts_tablet_manager.cc:1428] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 14:44:24.424947 27359 tablet_bootstrap.cc:492] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea: No bootstrap required, opened a new log
I20260810 14:44:24.425140 27359 ts_tablet_manager.cc:1397] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260810 14:44:24.425774 27359 raft_consensus.cc:348] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.426013 27359 raft_consensus.cc:374] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:24.426119 27359 raft_consensus.cc:729] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fae9e09726244718a664bda598cef4ea, State: Initialized, Role: FOLLOWER
I20260810 14:44:24.426393 27359 consensus_queue.cc:260] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.427470 27363 tablet_replica.cc:439] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } } }
I20260810 14:44:24.427775 27363 tablet_replica.cc:442] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:44:24.428208 27359 ts_tablet_manager.cc:1428] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260810 14:44:24.829805 27361 raft_consensus.cc:482] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:44:24.830096 27361 raft_consensus.cc:504] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.831785 27361 leader_election.cc:288] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fae9e09726244718a664bda598cef4ea (127.24.215.130:33359), d248832de2f24955bd5f623cc914eecd (127.24.215.131:34351)
I20260810 14:44:24.849045 27362 raft_consensus.cc:482] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:44:24.849303 27362 raft_consensus.cc:504] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.850085 27362 leader_election.cc:288] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 722f1ef69fc84c7880bfca29f3910958 (127.24.215.129:42511), fae9e09726244718a664bda598cef4ea (127.24.215.130:33359)
I20260810 14:44:24.850651 27109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36ce25c1d4b4ccca5599ce7b426ff1e" candidate_uuid: "d248832de2f24955bd5f623cc914eecd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae9e09726244718a664bda598cef4ea" is_pre_election: true
I20260810 14:44:24.850697 26949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36ce25c1d4b4ccca5599ce7b426ff1e" candidate_uuid: "d248832de2f24955bd5f623cc914eecd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "722f1ef69fc84c7880bfca29f3910958" is_pre_election: true
I20260810 14:44:24.850927 27109 raft_consensus.cc:2454] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d248832de2f24955bd5f623cc914eecd in term 0.
I20260810 14:44:24.851027 26949 raft_consensus.cc:2454] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d248832de2f24955bd5f623cc914eecd in term 0.
I20260810 14:44:24.851469 27197 leader_election.cc:302] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 722f1ef69fc84c7880bfca29f3910958, d248832de2f24955bd5f623cc914eecd; no voters:
I20260810 14:44:24.851787 27362 raft_consensus.cc:2781] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:44:24.851902 27362 raft_consensus.cc:482] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:44:24.852007 27362 raft_consensus.cc:3037] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:24.854427 27362 raft_consensus.cc:504] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.855139 27404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.832112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:34351 (local address 127.24.215.129:39171)
0810 14:44:24.847386 (+ 15274us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:24.847418 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:24.847654 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:24.848181 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:24.848186 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:24.848246 (+ 60us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:24.848517 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.848528 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.849913 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.849921 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.852448 (+ 2527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.852462 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.854174 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.854180 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.854383 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.854421 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 14:44:24.854521 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 14:44:24.854658 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15079,"mutex_wait_us":75,"spinlock_wait_cycles":38656,"thread_start_us":15805,"threads_started":1}
I20260810 14:44:24.855635 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.832267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:46783 (local address 127.24.215.130:33359)
0810 14:44:24.847809 (+ 15542us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:24.847819 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:24.848460 (+ 641us) server_negotiation.cc:407] Connection header received
0810 14:44:24.848535 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:24.848538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:24.848595 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:24.848712 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:24.849383 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.850435 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.852997 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.854198 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.854362 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.854444 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:24.854814 (+ 370us) server_negotiation.cc:299] Negotiation successful
0810 14:44:24.854962 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15423,"thread_start_us":15301,"threads_started":1}
I20260810 14:44:24.855556 26949 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36ce25c1d4b4ccca5599ce7b426ff1e" candidate_uuid: "d248832de2f24955bd5f623cc914eecd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "722f1ef69fc84c7880bfca29f3910958"
I20260810 14:44:24.855695 27406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.832267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:39171 (local address 127.24.215.131:34351)
0810 14:44:24.847404 (+ 15137us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:24.847415 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:24.847865 (+ 450us) server_negotiation.cc:407] Connection header received
0810 14:44:24.847944 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:24.847949 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:24.848014 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:24.848151 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:24.848693 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.849752 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.852630 (+ 2878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.853823 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.853998 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.854074 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:24.855227 (+ 1153us) server_negotiation.cc:299] Negotiation successful
0810 14:44:24.855373 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14907,"thread_start_us":15402,"threads_started":1}
I20260810 14:44:24.855806 26949 raft_consensus.cc:3037] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:24.856024 27109 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36ce25c1d4b4ccca5599ce7b426ff1e" candidate_uuid: "d248832de2f24955bd5f623cc914eecd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae9e09726244718a664bda598cef4ea"
I20260810 14:44:24.856166 27273 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36ce25c1d4b4ccca5599ce7b426ff1e" candidate_uuid: "722f1ef69fc84c7880bfca29f3910958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d248832de2f24955bd5f623cc914eecd" is_pre_election: true
I20260810 14:44:24.856279 27109 raft_consensus.cc:3037] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:24.856472 27273 raft_consensus.cc:2379] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 722f1ef69fc84c7880bfca29f3910958 in current term 1: Already voted for candidate d248832de2f24955bd5f623cc914eecd in this term.
I20260810 14:44:24.857048 27402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.832112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:33359 (local address 127.24.215.129:46783)
0810 14:44:24.848211 (+ 16099us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:24.848234 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:24.848364 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:24.848735 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:24.848739 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:24.848763 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:24.849149 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.849163 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.850611 (+ 1448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.850620 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.852800 (+ 2180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.852815 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.854442 (+ 1627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.854451 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.854642 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.854678 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 14:44:24.854779 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 14:44:24.854927 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15969,"mutex_wait_us":47,"thread_start_us":15947,"threads_started":1}
I20260810 14:44:24.857789 27362 leader_election.cc:288] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [CANDIDATE]: Term 1 election: Requested vote from peers 722f1ef69fc84c7880bfca29f3910958 (127.24.215.129:42511), fae9e09726244718a664bda598cef4ea (127.24.215.130:33359)
I20260810 14:44:24.857795 27108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c36ce25c1d4b4ccca5599ce7b426ff1e" candidate_uuid: "722f1ef69fc84c7880bfca29f3910958" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fae9e09726244718a664bda598cef4ea" is_pre_election: true
I20260810 14:44:24.858692 26869 leader_election.cc:302] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 722f1ef69fc84c7880bfca29f3910958; no voters: d248832de2f24955bd5f623cc914eecd, fae9e09726244718a664bda598cef4ea
I20260810 14:44:24.858990 26949 raft_consensus.cc:2454] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d248832de2f24955bd5f623cc914eecd in term 1.
I20260810 14:44:24.859205 27109 raft_consensus.cc:2454] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d248832de2f24955bd5f623cc914eecd in term 1.
I20260810 14:44:24.859193 27361 raft_consensus.cc:2726] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 14:44:24.859575 27197 leader_election.cc:302] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d248832de2f24955bd5f623cc914eecd, fae9e09726244718a664bda598cef4ea; no voters:
I20260810 14:44:24.859823 27362 raft_consensus.cc:2781] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:24.860317 27362 raft_consensus.cc:686] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [term 1 LEADER]: Becoming Leader. State: Replica: d248832de2f24955bd5f623cc914eecd, State: Running, Role: LEADER
I20260810 14:44:24.860635 27362 consensus_queue.cc:237] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } }
I20260810 14:44:24.862200 27407 tablet_replica.cc:439] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd: TxnStatusTablet state changed. Reason: New leader d248832de2f24955bd5f623cc914eecd. Latest consensus state: current_term: 1 leader_uuid: "d248832de2f24955bd5f623cc914eecd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } } }
I20260810 14:44:24.862510 27407 tablet_replica.cc:442] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:44:24.864001 27409 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 14:44:24.863926 26764 catalog_manager.cc:5495] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd reported cstate change: term changed from 0 to 1, leader changed from <none> to d248832de2f24955bd5f623cc914eecd (127.24.215.131). New cstate: current_term: 1 leader_uuid: "d248832de2f24955bd5f623cc914eecd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } health_report { overall_health: HEALTHY } } }
I20260810 14:44:24.864274 27409 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 14:44:24.883458 27406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.877605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45878 (local address 127.24.215.131:34351)
0810 14:44:24.877814 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:24.877820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:24.877840 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:44:24.878108 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:24.878112 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:24.878192 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:24.878334 (+ 142us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:24.878742 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.879640 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.881008 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.881529 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.881674 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.882159 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:24.882683 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:24.882885 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:24.883033 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 14:44:24.883159 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:44:24.883765 27396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.877417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:34351 (local address 127.0.0.1:45878)
0810 14:44:24.877677 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:24.877701 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:24.877853 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:24.878359 (+ 506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:24.878364 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:24.878376 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:44:24.878600 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.878610 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.879788 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.879793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.880849 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.880860 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.881689 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.881695 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.881884 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.881947 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:44:24.882875 (+ 928us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:44:24.882903 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 14:44:24.883005 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 14:44:24.883207 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 14:44:24.889559 27109 raft_consensus.cc:1261] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea [term 1 FOLLOWER]: Refusing update from remote peer d248832de2f24955bd5f623cc914eecd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:24.889763 26949 raft_consensus.cc:1261] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958 [term 1 FOLLOWER]: Refusing update from remote peer d248832de2f24955bd5f623cc914eecd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:24.890503 27407 consensus_queue.cc:1035] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:24.891037 27362 consensus_queue.cc:1035] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd [LEADER]: Connected to new peer: Peer: permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:24.894917 27361 tablet_replica.cc:439] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958: TxnStatusTablet state changed. Reason: New leader d248832de2f24955bd5f623cc914eecd. Latest consensus state: current_term: 1 leader_uuid: "d248832de2f24955bd5f623cc914eecd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } } }
I20260810 14:44:24.895288 27361 tablet_replica.cc:442] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:44:24.898270 27413 mvcc.cc:205] Tried to move back new op lower bound from 7316984073779138560 to 7316984073671446528. Current Snapshot: MvccSnapshot[applied={T|T < 7316984073779138560}]
I20260810 14:44:24.902196 27407 tablet_replica.cc:439] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d248832de2f24955bd5f623cc914eecd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } } }
I20260810 14:44:24.903613 27407 tablet_replica.cc:442] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:44:24.904649 27412 mvcc.cc:205] Tried to move back new op lower bound from 7316984073779138560 to 7316984073671446528. Current Snapshot: MvccSnapshot[applied={T|T < 7316984073779138560 or (T in {7316984073779138560})}]
I20260810 14:44:24.910589 27361 tablet_replica.cc:439] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d248832de2f24955bd5f623cc914eecd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } } }
I20260810 14:44:24.910842 27361 tablet_replica.cc:442] T c36ce25c1d4b4ccca5599ce7b426ff1e P 722f1ef69fc84c7880bfca29f3910958: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:44:24.920502 27406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.914041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45884 (local address 127.24.215.131:34351)
0810 14:44:24.914307 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:24.914317 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:24.914441 (+ 124us) server_negotiation.cc:407] Connection header received
0810 14:44:24.914721 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:24.914727 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:24.914807 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:24.914869 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:24.915685 (+ 816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.916835 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.918510 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.918993 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.919163 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.919577 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:24.919728 (+ 151us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:24.919832 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:24.919989 (+ 157us) server_negotiation.cc:299] Negotiation successful
0810 14:44:24.920070 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 14:44:24.925961 27407 consensus_queue.cc:1035] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [LEADER]: Connected to new peer: Peer: permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:24.934496 27407 consensus_queue.cc:1035] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [LEADER]: Connected to new peer: Peer: permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:24.968942 27363 tablet_replica.cc:439] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea: TxnStatusTablet state changed. Reason: New leader d248832de2f24955bd5f623cc914eecd. Latest consensus state: current_term: 1 leader_uuid: "d248832de2f24955bd5f623cc914eecd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } } }
I20260810 14:44:24.969285 27363 tablet_replica.cc:442] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:44:24.970479 27407 tablet_replica.cc:439] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d248832de2f24955bd5f623cc914eecd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } } }
I20260810 14:44:24.970821 27407 tablet_replica.cc:442] T c36ce25c1d4b4ccca5599ce7b426ff1e P d248832de2f24955bd5f623cc914eecd: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:44:24.980146 27406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.947390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:55097 (local address 127.24.215.131:34351)
0810 14:44:24.947636 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:24.947646 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:24.960101 (+ 12455us) server_negotiation.cc:407] Connection header received
0810 14:44:24.960337 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:24.960341 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:24.960439 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:24.960518 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:24.961226 (+ 708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.962429 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.965254 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.965946 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.966051 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.979011 (+ 12960us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:24.979224 (+ 213us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:24.979314 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:24.979741 (+ 427us) server_negotiation.cc:299] Negotiation successful
0810 14:44:24.979857 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 14:44:24.981168 27428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:24.946989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:34351 (local address 127.24.215.131:55097)
0810 14:44:24.959996 (+ 13007us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:24.960020 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:24.960207 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:24.960710 (+ 503us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:24.960714 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:24.960726 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:44:24.961053 (+ 327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.961068 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.962622 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.962627 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.964599 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:24.964615 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:24.966259 (+ 1644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:24.966266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:24.966495 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:24.966572 (+ 77us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:44:24.979447 (+ 12875us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:44:24.979456 (+ 9us) client_negotiation.cc:769] Sending connection context
0810 14:44:24.980448 (+ 992us) client_negotiation.cc:240] Negotiation successful
0810 14:44:24.980628 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12772,"thread_start_us":12907,"threads_started":1}
I20260810 14:44:24.984289 27363 tablet_replica.cc:439] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d248832de2f24955bd5f623cc914eecd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "722f1ef69fc84c7880bfca29f3910958" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42511 } } peers { permanent_uuid: "fae9e09726244718a664bda598cef4ea" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33359 } } peers { permanent_uuid: "d248832de2f24955bd5f623cc914eecd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34351 } } }
I20260810 14:44:24.984594 27363 tablet_replica.cc:442] T c36ce25c1d4b4ccca5599ce7b426ff1e P fae9e09726244718a664bda598cef4ea: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:44:25.093961 26762 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:44:25.094283 26762 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44086:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:44:25.098789 26762 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f6e26ec58524e3f86bc564ddfd0e893: Sending DeleteTablet for 3 replicas of tablet 13b38983588a4df5a0eacb18d589b399
I20260810 14:44:25.099799 27088 tablet_service.cc:1514] Processing DeleteTablet for tablet 13b38983588a4df5a0eacb18d589b399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:25 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:33208
I20260810 14:44:25.099797 26927 tablet_service.cc:1514] Processing DeleteTablet for tablet 13b38983588a4df5a0eacb18d589b399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:25 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:34952
I20260810 14:44:25.099817 27251 tablet_service.cc:1514] Processing DeleteTablet for tablet 13b38983588a4df5a0eacb18d589b399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:25 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:45862
I20260810 14:44:25.115262 27437 tablet_replica.cc:330] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd: stopping tablet replica
I20260810 14:44:25.116267 27437 raft_consensus.cc:2229] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:44:25.117259 27436 tablet_replica.cc:330] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea: stopping tablet replica
I20260810 14:44:25.117383 27437 raft_consensus.cc:2258] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:25.117653 27436 raft_consensus.cc:2229] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:44:25.118040 27436 raft_consensus.cc:2258] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:25.119437 27436 ts_tablet_manager.cc:1905] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:25.120108 27437 ts_tablet_manager.cc:1905] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:25.125649 27435 tablet_replica.cc:330] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958: stopping tablet replica
I20260810 14:44:25.125864 27435 raft_consensus.cc:2229] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:44:25.126142 27435 raft_consensus.cc:2258] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:25.127318 27435 ts_tablet_manager.cc:1905] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:25.127904 27437 ts_tablet_manager.cc:1918] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:44:25.128055 27437 log.cc:1196] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/13b38983588a4df5a0eacb18d589b399
I20260810 14:44:25.128572 27437 ts_tablet_manager.cc:1939] T 13b38983588a4df5a0eacb18d589b399 P d248832de2f24955bd5f623cc914eecd: Deleting consensus metadata
I20260810 14:44:25.129918 26744 catalog_manager.cc:4841] TS d248832de2f24955bd5f623cc914eecd (127.24.215.131:34351): tablet 13b38983588a4df5a0eacb18d589b399 (table test-table [id=a757cf518b794f7e8015fb6a5eaa5546]) successfully deleted
I20260810 14:44:25.137583 27436 ts_tablet_manager.cc:1918] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:44:25.137847 27436 log.cc:1196] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/13b38983588a4df5a0eacb18d589b399
I20260810 14:44:25.138528 27436 ts_tablet_manager.cc:1939] T 13b38983588a4df5a0eacb18d589b399 P fae9e09726244718a664bda598cef4ea: Deleting consensus metadata
I20260810 14:44:25.140314 26745 catalog_manager.cc:4841] TS fae9e09726244718a664bda598cef4ea (127.24.215.130:33359): tablet 13b38983588a4df5a0eacb18d589b399 (table test-table [id=a757cf518b794f7e8015fb6a5eaa5546]) successfully deleted
I20260810 14:44:25.140528 27435 ts_tablet_manager.cc:1918] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:44:25.140740 27435 log.cc:1196] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/13b38983588a4df5a0eacb18d589b399
I20260810 14:44:25.141404 27435 ts_tablet_manager.cc:1939] T 13b38983588a4df5a0eacb18d589b399 P 722f1ef69fc84c7880bfca29f3910958: Deleting consensus metadata
I20260810 14:44:25.142753 26745 catalog_manager.cc:4841] TS 722f1ef69fc84c7880bfca29f3910958 (127.24.215.129:42511): tablet 13b38983588a4df5a0eacb18d589b399 (table test-table [id=a757cf518b794f7e8015fb6a5eaa5546]) successfully deleted
Aug 10 14:44:25 dist-test-slave-vkjp krb5kdc[26706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373063, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:44:25.154016 27441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:25.116648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34966 (local address 127.24.215.129:42511)
0810 14:44:25.133572 (+ 16924us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:25.133577 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:25.133599 (+ 22us) server_negotiation.cc:407] Connection header received
0810 14:44:25.133652 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:25.133656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:25.133725 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:25.133874 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:25.136059 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:25.137263 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:25.139197 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:25.139916 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:25.140043 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:25.145919 (+ 5876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:25.145940 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:25.145961 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:25.146015 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:25.150185 (+ 4170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:25.151475 (+ 1290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:25.151485 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:25.151490 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:25.151661 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:25.152116 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:25.152121 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:25.152125 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:25.152602 (+ 477us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:25.152892 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:25.153316 (+ 424us) server_negotiation.cc:299] Negotiation successful
0810 14:44:25.153602 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16792,"thread_start_us":16627,"threads_started":1}
W20260810 14:44:25.155182 27002 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34966
I20260810 14:44:25.192739 27444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:25.159429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44112 (local address 127.24.215.190:40341)
0810 14:44:25.172649 (+ 13220us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:25.172659 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:25.173355 (+ 696us) server_negotiation.cc:407] Connection header received
0810 14:44:25.173556 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:25.173562 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:25.173651 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:25.173779 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:25.175975 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:25.177262 (+ 1287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:25.179636 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:25.180374 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:25.180510 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:25.184126 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:25.184136 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:25.184141 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:25.184193 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:25.188824 (+ 4631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:25.189962 (+ 1138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:25.189969 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:25.189973 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:25.190159 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:25.190677 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:25.190685 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:25.190690 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:25.191207 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:25.191513 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:25.191914 (+ 401us) server_negotiation.cc:299] Negotiation successful
0810 14:44:25.192217 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13027,"thread_start_us":13213,"threads_started":1}
W20260810 14:44:25.193435 26762 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:44112
I20260810 14:44:25.196015 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 26848
I20260810 14:44:25.215859 27441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:25.196979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34980 (local address 127.24.215.129:42511)
0810 14:44:25.197534 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:25.197544 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:25.197775 (+ 231us) server_negotiation.cc:407] Connection header received
0810 14:44:25.198074 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:25.198082 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:25.198176 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:25.198255 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:25.200172 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:25.201443 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:25.203732 (+ 2289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:25.204476 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:25.204618 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:25.208032 (+ 3414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:25.208040 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:25.208046 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:25.208094 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:25.212241 (+ 4147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:25.213263 (+ 1022us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:25.213268 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:25.213272 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:25.213536 (+ 264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:25.214056 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:25.214062 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:25.214066 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:25.214449 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:25.214740 (+ 291us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:25.215214 (+ 474us) server_negotiation.cc:299] Negotiation successful
0810 14:44:25.215481 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372}
W20260810 14:44:25.216653 27002 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34980
W20260810 14:44:25.217420 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 26848: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:44:25.217649 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 26848
I20260810 14:44:25.252022 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 27011
Aug 10 14:44:25 dist-test-slave-vkjp krb5kdc[26706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373063, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:44:25.275141 27405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:25.253009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33230 (local address 127.24.215.130:33359)
0810 14:44:25.253260 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:25.253268 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:25.253294 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:44:25.253372 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:25.253380 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:25.253479 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:25.253609 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:25.255138 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:25.256472 (+ 1334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:25.258264 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:25.259115 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:25.259289 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:25.266235 (+ 6946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:25.266261 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:25.266284 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:25.266345 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:25.271339 (+ 4994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:25.272321 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:25.272335 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:25.272341 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:25.272539 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:25.273027 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:25.273034 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:25.273039 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:25.273579 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:25.273933 (+ 354us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:25.274316 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 14:44:25.274636 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
W20260810 14:44:25.275930 27165 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33230
W20260810 14:44:25.276832 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 27011: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:44:25.277036 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 27011
I20260810 14:44:25.312466 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 27177
Aug 10 14:44:25 dist-test-slave-vkjp krb5kdc[26706](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373063, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:44:25.335398 27406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:25.313618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45898 (local address 127.24.215.131:34351)
0810 14:44:25.313841 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:25.313850 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:25.313874 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:44:25.313965 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:25.313974 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:25.314083 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:25.314218 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:25.315995 (+ 1777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:25.317482 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:25.319581 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:25.320547 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:25.320708 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:25.326607 (+ 5899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:25.326621 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:25.326639 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:25.326695 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:25.331119 (+ 4424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:25.332172 (+ 1053us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:25.332185 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:25.332192 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:25.332410 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:25.332882 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:25.332890 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:25.332906 (+ 16us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:25.333538 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:25.334000 (+ 462us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:25.334226 (+ 226us) server_negotiation.cc:299] Negotiation successful
0810 14:44:25.334535 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 14:44:25.336285 27326 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45898
W20260810 14:44:25.337060 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 27177: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:44:25.337221 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 27177
I20260810 14:44:25.371449 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 26721
I20260810 14:44:25.372936 26809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:25.489368 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 26721
2026-08-10T14:44:25Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5068 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:44:25 dist-test-slave-vkjp krb5kdc[27449](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:44:25 dist-test-slave-vkjp krb5kdc[27449](info): set up 2 sockets
Aug 10 14:44:25 dist-test-slave-vkjp krb5kdc[27449](info): commencing operation
krb5kdc: starting...
W20260810 14:44:26.664494 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.001s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:44:26 dist-test-slave-vkjp krb5kdc[27449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:44:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:44:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:44:26.867473 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:41921
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:33741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:41921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:27.202977 27464 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:27.203302 27464 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:27.203397 27464 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:27.210516 27464 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:27.210628 27464 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 14:44:27.210670 27464 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:27.210731 27464 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:27.210783 27464 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:27.210839 27464 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:44:27.221410 27464 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:33741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:41921
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:41921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:27.222681 27464 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:27.223995 27464 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:27.235919 27473 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:27.235898 27469 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:27.236002 27471 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:27.237424 27464 server_base.cc:1034] running on GCE node
I20260810 14:44:27.238266 27464 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:27.239868 27464 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:27.241163 27464 hybrid_clock.cc:648] HybridClock initialized: now 1786373067241083 us; error 60 us; skew 500 ppm
I20260810 14:44:27.241523 27464 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:27 dist-test-slave-vkjp krb5kdc[27449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373067, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:27.244832 27464 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:44:27.246366 27464 webserver.cc:460] Webserver started at http://127.24.215.190:39253/ using document root <none> and password file <none>
I20260810 14:44:27.247207 27464 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:27.247301 27464 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:27.247576 27464 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:27.250784 27464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "0876ec41ef5742ecab95b58e40ceb03d"
format_stamp: "Formatted at 2026-08-10 14:44:27 on dist-test-slave-vkjp"
server_key: "e424259a9e19256222eed110c7988ae0"
server_key_iv: "02c88f1aba391228b7302901035ae3ca"
server_key_version: "encryptionkey@0"
I20260810 14:44:27.251627 27464 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "0876ec41ef5742ecab95b58e40ceb03d"
format_stamp: "Formatted at 2026-08-10 14:44:27 on dist-test-slave-vkjp"
server_key: "e424259a9e19256222eed110c7988ae0"
server_key_iv: "02c88f1aba391228b7302901035ae3ca"
server_key_version: "encryptionkey@0"
I20260810 14:44:27.257136 27464 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:44:27.260766 27480 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:27.262198 27464 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 14:44:27.262389 27464 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "0876ec41ef5742ecab95b58e40ceb03d"
format_stamp: "Formatted at 2026-08-10 14:44:27 on dist-test-slave-vkjp"
server_key: "e424259a9e19256222eed110c7988ae0"
server_key_iv: "02c88f1aba391228b7302901035ae3ca"
server_key_version: "encryptionkey@0"
I20260810 14:44:27.262630 27464 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:27.281576 27464 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:27.285908 27464 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:27.286213 27464 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:27.308904 27464 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:41921
I20260810 14:44:27.308934 27557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:41921 every 8 connection(s)
I20260810 14:44:27.310364 27464 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:44:27.314172 27558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:27.318289 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 27464
I20260810 14:44:27.318506 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:44:27.319357 25438 external_mini_cluster.cc:1428] Setting key ce0e0fb0b4330f4808c4fb3aedb2a0ca
I20260810 14:44:27.326164 27558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d: Bootstrap starting.
I20260810 14:44:27.329843 27558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:27.331132 27558 log.cc:824] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:27.334060 27558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d: No bootstrap required, opened a new log
Aug 10 14:44:27 dist-test-slave-vkjp krb5kdc[27449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373066, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:27.343549 27558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0876ec41ef5742ecab95b58e40ceb03d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41921 } }
I20260810 14:44:27.343925 27558 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:27.344010 27561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:27.321534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50564 (local address 127.24.215.190:41921)
0810 14:44:27.322330 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:27.322342 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:27.322384 (+ 42us) server_negotiation.cc:407] Connection header received
0810 14:44:27.324593 (+ 2209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:27.324617 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:27.325015 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:27.325337 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:27.327236 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:27.328907 (+ 1671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:27.330686 (+ 1779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:27.331500 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:27.331691 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:27.336221 (+ 4530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:27.336232 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:27.336247 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:27.336322 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:27.339672 (+ 3350us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:27.340524 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:27.340531 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:27.340537 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:27.340707 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:27.341129 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:27.341133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:27.341135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:27.341717 (+ 582us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:27.341943 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:27.342458 (+ 515us) server_negotiation.cc:299] Negotiation successful
0810 14:44:27.342874 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":380,"threads_started":1}
I20260810 14:44:27.344060 27558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0876ec41ef5742ecab95b58e40ceb03d, State: Initialized, Role: FOLLOWER
I20260810 14:44:27.344717 27558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0876ec41ef5742ecab95b58e40ceb03d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41921 } }
I20260810 14:44:27.344987 27558 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:44:27.345144 27558 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:44:27.345360 27558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:27.348316 27558 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0876ec41ef5742ecab95b58e40ceb03d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41921 } }
I20260810 14:44:27.348776 27558 leader_election.cc:302] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0876ec41ef5742ecab95b58e40ceb03d; no voters:
I20260810 14:44:27.349289 27558 leader_election.cc:288] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:44:27.349495 27563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:27.350946 27563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [term 1 LEADER]: Becoming Leader. State: Replica: 0876ec41ef5742ecab95b58e40ceb03d, State: Running, Role: LEADER
I20260810 14:44:27.351294 27558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:44:27.351578 27563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0876ec41ef5742ecab95b58e40ceb03d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41921 } }
I20260810 14:44:27.354528 27564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0876ec41ef5742ecab95b58e40ceb03d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0876ec41ef5742ecab95b58e40ceb03d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41921 } } }
I20260810 14:44:27.354921 27564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:27.355368 27565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0876ec41ef5742ecab95b58e40ceb03d. Latest consensus state: current_term: 1 leader_uuid: "0876ec41ef5742ecab95b58e40ceb03d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0876ec41ef5742ecab95b58e40ceb03d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41921 } } }
I20260810 14:44:27.355696 27565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:27.359862 27572 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:44:27.365231 27572 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:44:27.373950 27572 catalog_manager.cc:1347] Generated new cluster ID: 6d4c4f303b7545eeadcebd5d649145a1
I20260810 14:44:27.374069 27572 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:44:27.399683 27572 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:44:27.401290 27572 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:44:27.415581 27572 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d: Generated new TSK 0
I20260810 14:44:27.416531 27572 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:44:27.474444 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:41921
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:33741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:27.816074 27585 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:27.816359 27585 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:27.816540 27585 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:27.823776 27585 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:27.823884 27585 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:27.824038 27585 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 14:44:27.824119 27585 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:44:27.824260 27585 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:44:27.834022 27585 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:33741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.215.190:41921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:27.835378 27585 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:27.836694 27585 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:27.850180 27595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:27.850281 27593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:27.850253 27592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:27.851145 27585 server_base.cc:1034] running on GCE node
I20260810 14:44:27.851877 27585 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:27.853024 27585 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:27.854259 27585 hybrid_clock.cc:648] HybridClock initialized: now 1786373067854181 us; error 71 us; skew 500 ppm
I20260810 14:44:27.854595 27585 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:27 dist-test-slave-vkjp krb5kdc[27449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373067, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:27.862746 27585 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:44:27.864550 27585 webserver.cc:460] Webserver started at http://127.24.215.129:33529/ using document root <none> and password file <none>
I20260810 14:44:27.865289 27585 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:27.865458 27585 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:27.865792 27585 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:27.869163 27585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "3d013baa9b7d445c8a956138221b53ff"
format_stamp: "Formatted at 2026-08-10 14:44:27 on dist-test-slave-vkjp"
server_key: "a40c27c6149404b8cbd6b71e8f13dd8b"
server_key_iv: "b844b500cab3297312d6483e8f00c222"
server_key_version: "encryptionkey@0"
I20260810 14:44:27.869995 27585 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "3d013baa9b7d445c8a956138221b53ff"
format_stamp: "Formatted at 2026-08-10 14:44:27 on dist-test-slave-vkjp"
server_key: "a40c27c6149404b8cbd6b71e8f13dd8b"
server_key_iv: "b844b500cab3297312d6483e8f00c222"
server_key_version: "encryptionkey@0"
I20260810 14:44:27.875696 27585 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260810 14:44:27.879413 27602 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:27.880888 27585 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:44:27.881105 27585 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "3d013baa9b7d445c8a956138221b53ff"
format_stamp: "Formatted at 2026-08-10 14:44:27 on dist-test-slave-vkjp"
server_key: "a40c27c6149404b8cbd6b71e8f13dd8b"
server_key_iv: "b844b500cab3297312d6483e8f00c222"
server_key_version: "encryptionkey@0"
I20260810 14:44:27.881340 27585 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:27.897749 27585 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:27.902144 27585 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:27.902475 27585 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:27.905333 27585 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:27.905471 27585 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:27.905627 27585 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:27.905731 27585 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:44:27 dist-test-slave-vkjp krb5kdc[27449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373067, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:27.931903 27611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:27.906290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:50703 (local address 127.24.215.190:41921)
0810 14:44:27.906803 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:27.906810 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:27.908120 (+ 1310us) server_negotiation.cc:407] Connection header received
0810 14:44:27.911442 (+ 3322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:27.911450 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:27.911560 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:27.911684 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:27.914055 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:27.915489 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:27.918891 (+ 3402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:27.919515 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:27.919679 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:27.925236 (+ 5557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:27.925241 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:27.925245 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:27.925279 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:27.928242 (+ 2963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:27.929048 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:27.929058 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:27.929064 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:27.929268 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:27.929738 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:27.929746 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:27.929751 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:27.930200 (+ 449us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:27.930376 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:27.931119 (+ 743us) server_negotiation.cc:299] Negotiation successful
0810 14:44:27.931396 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":259,"threads_started":1}
I20260810 14:44:27.933008 27612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:27.906943 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41921 (local address 127.24.215.129:50703)
0810 14:44:27.907915 (+ 972us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:27.907978 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:27.910991 (+ 3013us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:27.912052 (+ 1061us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:27.912065 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:27.912945 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:27.913810 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:27.913855 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:27.915702 (+ 1847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:27.915709 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:27.918714 (+ 3005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:27.918736 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:27.919892 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:27.919908 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:27.920315 (+ 407us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:27.921238 (+ 923us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:27.921281 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:27.924980 (+ 3699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:27.928434 (+ 3454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:27.928441 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:27.928455 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:27.928910 (+ 455us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:27.929420 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:27.929423 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:27.929426 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:27.929622 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:27.930343 (+ 721us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:27.930349 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:27.930710 (+ 361us) client_negotiation.cc:769] Sending connection context
0810 14:44:27.930954 (+ 244us) client_negotiation.cc:240] Negotiation successful
0810 14:44:27.931627 (+ 673us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":670,"thread_start_us":416,"threads_started":1}
I20260810 14:44:27.954514 27611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:27.939410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:57089 (local address 127.24.215.190:41921)
0810 14:44:27.939640 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:27.939648 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:27.940260 (+ 612us) server_negotiation.cc:407] Connection header received
0810 14:44:27.940409 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:27.940419 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:27.940546 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:27.940680 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:27.941659 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:27.943180 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:27.945250 (+ 2070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:27.945662 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:27.945813 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:27.947961 (+ 2148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:27.947967 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:27.947970 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:27.948002 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:27.951377 (+ 3375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:27.952483 (+ 1106us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:27.952487 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:27.952490 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:27.952628 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:27.953078 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:27.953082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:27.953084 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:27.953389 (+ 305us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:27.953507 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:27.954072 (+ 565us) server_negotiation.cc:299] Negotiation successful
0810 14:44:27.954247 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 14:44:27.955148 27692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:27.939492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41921 (local address 127.24.215.129:57089)
0810 14:44:27.940157 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:27.940175 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:27.940285 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:27.940733 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:27.940737 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:27.941215 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:27.941512 (+ 297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:27.941521 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:27.943353 (+ 1832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:27.943360 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:27.945050 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:27.945068 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:27.945866 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:27.945872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:27.946045 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:27.946689 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:27.946707 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:27.947692 (+ 985us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:27.951530 (+ 3838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:27.951539 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:27.951546 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:27.952300 (+ 754us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:27.952747 (+ 447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:27.952755 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:27.952761 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:27.952960 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:27.953511 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:27.953518 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:27.953789 (+ 271us) client_negotiation.cc:769] Sending connection context
0810 14:44:27.953981 (+ 192us) client_negotiation.cc:240] Negotiation successful
0810 14:44:27.954452 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":352,"threads_started":1}
I20260810 14:44:27.969101 27585 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:45257
I20260810 14:44:27.969135 27743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:45257 every 8 connection(s)
I20260810 14:44:27.970420 27585 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:44:27.972739 27744 heartbeater.cc:347] Connected to a master server at 127.24.215.190:41921
I20260810 14:44:27.972991 27744 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:27.973675 27744 heartbeater.cc:511] Master 127.24.215.190:41921 requested a full tablet report, sending...
I20260810 14:44:27.975752 27505 ts_manager.cc:194] Registered new tserver with Master: 3d013baa9b7d445c8a956138221b53ff (127.24.215.129:45257)
I20260810 14:44:27.976004 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 27585
I20260810 14:44:27.976207 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:44:27.976835 25438 external_mini_cluster.cc:1428] Setting key 8e260dec3ebe2e92e1fc9d34a539f7a1
I20260810 14:44:27.977433 27505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:50703
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:44:28.023423 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:41921
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:33741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:28.362142 27748 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:28.362455 27748 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:28.362649 27748 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:28.370229 27748 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:28.370339 27748 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:28.370518 27748 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 14:44:28.370600 27748 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:44:28.370734 27748 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:44:28.383875 27748 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:33741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.215.190:41921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:28.385710 27748 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:28.387884 27748 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:28.405020 27757 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:28.405718 27754 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:28.405822 27755 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:28.406205 27748 server_base.cc:1034] running on GCE node
I20260810 14:44:28.407045 27748 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:28.408412 27748 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:28.409701 27748 hybrid_clock.cc:648] HybridClock initialized: now 1786373068409658 us; error 102 us; skew 500 ppm
I20260810 14:44:28.410163 27748 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:28 dist-test-slave-vkjp krb5kdc[27449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373068, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:28.414767 27748 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:44:28.416803 27748 webserver.cc:460] Webserver started at http://127.24.215.130:40595/ using document root <none> and password file <none>
I20260810 14:44:28.417912 27748 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:28.418087 27748 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:28.418483 27748 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:28.423074 27748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "1f2682bb26fc42ac8a2f39e943a79522"
format_stamp: "Formatted at 2026-08-10 14:44:28 on dist-test-slave-vkjp"
server_key: "4697510d348b63db3d901b539e8bf7cc"
server_key_iv: "65295cbc8b2f18b77ab6ae5bc83bfec9"
server_key_version: "encryptionkey@0"
I20260810 14:44:28.424324 27748 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "1f2682bb26fc42ac8a2f39e943a79522"
format_stamp: "Formatted at 2026-08-10 14:44:28 on dist-test-slave-vkjp"
server_key: "4697510d348b63db3d901b539e8bf7cc"
server_key_iv: "65295cbc8b2f18b77ab6ae5bc83bfec9"
server_key_version: "encryptionkey@0"
I20260810 14:44:28.432983 27748 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.008s sys 0.000s
I20260810 14:44:28.438620 27768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:28.440383 27748 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 14:44:28.440548 27748 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "1f2682bb26fc42ac8a2f39e943a79522"
format_stamp: "Formatted at 2026-08-10 14:44:28 on dist-test-slave-vkjp"
server_key: "4697510d348b63db3d901b539e8bf7cc"
server_key_iv: "65295cbc8b2f18b77ab6ae5bc83bfec9"
server_key_version: "encryptionkey@0"
I20260810 14:44:28.440734 27748 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:28.461931 27748 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:28.466277 27748 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:28.466567 27748 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:28.469485 27748 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:28.469620 27748 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:28.469766 27748 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:28.469866 27748 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:44:28 dist-test-slave-vkjp krb5kdc[27449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373068, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:28.498170 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:28.470384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:33705 (local address 127.24.215.190:41921)
0810 14:44:28.471552 (+ 1168us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:28.471559 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:28.472615 (+ 1056us) server_negotiation.cc:407] Connection header received
0810 14:44:28.475165 (+ 2550us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:28.475173 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:28.475268 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:28.475399 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:28.478478 (+ 3079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:28.479839 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:28.482924 (+ 3085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:28.483539 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:28.483703 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:28.489327 (+ 5624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:28.489337 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:28.489342 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:28.489398 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:28.493775 (+ 4377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:28.495222 (+ 1447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:28.495232 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:28.495238 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:28.495478 (+ 240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:28.496068 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:28.496074 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:28.496078 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:28.496401 (+ 323us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:28.496590 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:28.497445 (+ 855us) server_negotiation.cc:299] Negotiation successful
0810 14:44:28.497717 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1018,"thread_start_us":908,"threads_started":1}
I20260810 14:44:28.499718 27780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:28.471320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41921 (local address 127.24.215.130:33705)
0810 14:44:28.472398 (+ 1078us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:28.472469 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:28.474594 (+ 2125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:28.476095 (+ 1501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:28.476112 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:28.477121 (+ 1009us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:28.478211 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:28.478243 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:28.480097 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:28.480105 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:28.482720 (+ 2615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:28.482739 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:28.483838 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:28.483848 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:28.484087 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:28.484885 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:28.484916 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:28.489001 (+ 4085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:28.494079 (+ 5078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:28.494100 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:28.494131 (+ 31us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:28.495039 (+ 908us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:28.495663 (+ 624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:28.495667 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:28.495670 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:28.495920 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:28.496566 (+ 646us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:28.496576 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:28.497012 (+ 436us) client_negotiation.cc:769] Sending connection context
0810 14:44:28.497349 (+ 337us) client_negotiation.cc:240] Negotiation successful
0810 14:44:28.497971 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"thread_start_us":452,"threads_started":1}
I20260810 14:44:28.523716 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:28.507406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:58845 (local address 127.24.215.190:41921)
0810 14:44:28.507628 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:28.507637 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:28.508218 (+ 581us) server_negotiation.cc:407] Connection header received
0810 14:44:28.508358 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:28.508367 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:28.508476 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:28.508605 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:28.509546 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:28.511017 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:28.512414 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:28.513176 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:28.513341 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:28.516030 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:28.516040 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:28.516048 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:28.516102 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:28.519800 (+ 3698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:28.520871 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:28.520878 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:28.520881 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:28.521083 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:28.521602 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:28.521611 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:28.521617 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:28.522253 (+ 636us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:28.522462 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:28.522927 (+ 465us) server_negotiation.cc:299] Negotiation successful
0810 14:44:28.523288 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 14:44:28.523952 27862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:28.507002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41921 (local address 127.24.215.130:58845)
0810 14:44:28.508087 (+ 1085us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:28.508109 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:28.508243 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:28.508640 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:28.508644 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:28.509104 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:28.509397 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:28.509406 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:28.511229 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:28.511233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:28.512266 (+ 1033us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:28.512279 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:28.513431 (+ 1152us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:28.513441 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:28.513644 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:28.514462 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:28.514493 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:28.515772 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:28.519964 (+ 4192us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:28.519970 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:28.519978 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:28.520637 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:28.521222 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:28.521227 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:28.521231 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:28.521446 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:28.522433 (+ 987us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:28.522437 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:28.522632 (+ 195us) client_negotiation.cc:769] Sending connection context
0810 14:44:28.522771 (+ 139us) client_negotiation.cc:240] Negotiation successful
0810 14:44:28.523159 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":935,"thread_start_us":790,"threads_started":1}
I20260810 14:44:28.530433 27748 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:36427
I20260810 14:44:28.530568 27907 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:36427 every 8 connection(s)
I20260810 14:44:28.531804 27748 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:44:28.534066 27908 heartbeater.cc:347] Connected to a master server at 127.24.215.190:41921
I20260810 14:44:28.534310 27908 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:28.535060 27908 heartbeater.cc:511] Master 127.24.215.190:41921 requested a full tablet report, sending...
I20260810 14:44:28.536432 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 27748
I20260810 14:44:28.536636 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:44:28.536860 27503 ts_manager.cc:194] Registered new tserver with Master: 1f2682bb26fc42ac8a2f39e943a79522 (127.24.215.130:36427)
I20260810 14:44:28.537122 25438 external_mini_cluster.cc:1428] Setting key 6cbd7b271ea149f117ba3179b4a1dde6
I20260810 14:44:28.538040 27503 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:33705
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:44:28.582610 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:41921
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:33741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:28.921708 27913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:28.922077 27913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:28.922264 27913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:28.929922 27913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:28.930097 27913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:28.930263 27913 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260810 14:44:28.930364 27913 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:44:28.930486 27913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:44:28.941313 27913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:33741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.215.190:41921
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:28.942684 27913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:28.944204 27913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:28.958426 27919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:28.958518 27918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:28.958472 27923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:28.959411 27913 server_base.cc:1034] running on GCE node
I20260810 14:44:28.960089 27913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:28.961155 27913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:28.962394 27913 hybrid_clock.cc:648] HybridClock initialized: now 1786373068962318 us; error 64 us; skew 500 ppm
I20260810 14:44:28.962743 27913 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:28 dist-test-slave-vkjp krb5kdc[27449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373068, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:28.966032 27913 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:44:28.967967 27913 webserver.cc:460] Webserver started at http://127.24.215.131:43561/ using document root <none> and password file <none>
I20260810 14:44:28.968967 27913 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:28.969110 27913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:28.969511 27913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:28.973649 27913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb"
format_stamp: "Formatted at 2026-08-10 14:44:28 on dist-test-slave-vkjp"
server_key: "138b97bc76021b83e8642dac98fabc3a"
server_key_iv: "4d2e55b74f253b8cc32f2941bc90e669"
server_key_version: "encryptionkey@0"
I20260810 14:44:28.974740 27913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb"
format_stamp: "Formatted at 2026-08-10 14:44:28 on dist-test-slave-vkjp"
server_key: "138b97bc76021b83e8642dac98fabc3a"
server_key_iv: "4d2e55b74f253b8cc32f2941bc90e669"
server_key_version: "encryptionkey@0"
I20260810 14:44:28.980935 27744 heartbeater.cc:503] Master 127.24.215.190:41921 was elected leader, sending a full tablet report...
I20260810 14:44:28.980953 27913 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260810 14:44:28.985916 27935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:28.987538 27913 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260810 14:44:28.987748 27913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb"
format_stamp: "Formatted at 2026-08-10 14:44:28 on dist-test-slave-vkjp"
server_key: "138b97bc76021b83e8642dac98fabc3a"
server_key_iv: "4d2e55b74f253b8cc32f2941bc90e669"
server_key_version: "encryptionkey@0"
I20260810 14:44:28.987985 27913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:29.000872 27913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:29.005448 27913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:29.005765 27913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:29.008565 27913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:29.008719 27913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:29.008858 27913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:29.008970 27913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:44:29 dist-test-slave-vkjp krb5kdc[27449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373068, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:29.036056 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.009299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:38849 (local address 127.24.215.190:41921)
0810 14:44:29.009525 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:29.009532 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:29.011044 (+ 1512us) server_negotiation.cc:407] Connection header received
0810 14:44:29.013828 (+ 2784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:29.013839 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:29.013980 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:29.014119 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:29.016544 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.017908 (+ 1364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.020645 (+ 2737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.021527 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.021697 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.026234 (+ 4537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:29.026244 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:29.026250 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:29.026302 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:29.030644 (+ 4342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:29.031923 (+ 1279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:29.031929 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:29.031939 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:29.032091 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:29.032735 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:29.032746 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:29.032749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:29.033194 (+ 445us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:29.033374 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:29.035189 (+ 1815us) server_negotiation.cc:299] Negotiation successful
0810 14:44:29.035507 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260810 14:44:29.037045 27944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.009911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41921 (local address 127.24.215.131:38849)
0810 14:44:29.010836 (+ 925us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:29.010885 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:29.013422 (+ 2537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:29.014428 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:29.014445 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:29.015369 (+ 924us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:29.016297 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.016331 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.018100 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.018111 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.020446 (+ 2335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.020470 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.021766 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.021772 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.021977 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.022742 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:29.022765 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:29.025891 (+ 3126us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:29.030849 (+ 4958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:29.030861 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:29.030875 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:29.031756 (+ 881us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:29.032221 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:29.032228 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:29.032235 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:29.032590 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:29.033352 (+ 762us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:29.033361 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:29.034573 (+ 1212us) client_negotiation.cc:769] Sending connection context
0810 14:44:29.034990 (+ 417us) client_negotiation.cc:240] Negotiation successful
0810 14:44:29.035715 (+ 725us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"thread_start_us":402,"threads_started":1}
I20260810 14:44:29.062251 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.046000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:37857 (local address 127.24.215.190:41921)
0810 14:44:29.046240 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:29.046247 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:29.047161 (+ 914us) server_negotiation.cc:407] Connection header received
0810 14:44:29.047460 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:29.047466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:29.047553 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:29.047677 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:29.049372 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.050426 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.052356 (+ 1930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.052993 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.053133 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.055761 (+ 2628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:29.055769 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:29.055773 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:29.055817 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:29.058800 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:29.059675 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:29.059683 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:29.059688 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:29.059925 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:29.060524 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:29.060529 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:29.060533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:29.060954 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:29.061140 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:29.061822 (+ 682us) server_negotiation.cc:299] Negotiation successful
0810 14:44:29.061993 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260810 14:44:29.063450 28033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.046301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41921 (local address 127.24.215.131:37857)
0810 14:44:29.047014 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:29.047039 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:29.047299 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:29.047878 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:29.047885 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:29.048641 (+ 756us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:29.049137 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.049154 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.050648 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.050652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.052194 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.052208 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.053192 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.053204 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.053436 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.054174 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:29.054206 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:29.055487 (+ 1281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:29.058942 (+ 3455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:29.058948 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:29.058952 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:29.059501 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:29.060071 (+ 570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:29.060079 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:29.060082 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:29.060278 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:29.061209 (+ 931us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:29.061212 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:29.061493 (+ 281us) client_negotiation.cc:769] Sending connection context
0810 14:44:29.062472 (+ 979us) client_negotiation.cc:240] Negotiation successful
0810 14:44:29.062944 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":352,"threads_started":1}
I20260810 14:44:29.064172 27913 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:37949
I20260810 14:44:29.064194 28077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:37949 every 8 connection(s)
I20260810 14:44:29.065541 27913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:44:29.067978 28078 heartbeater.cc:347] Connected to a master server at 127.24.215.190:41921
I20260810 14:44:29.068228 28078 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:29.068933 28078 heartbeater.cc:511] Master 127.24.215.190:41921 requested a full tablet report, sending...
I20260810 14:44:29.070452 27505 ts_manager.cc:194] Registered new tserver with Master: c4fca73b7f7b4e3ca3f5348bbba63fcb (127.24.215.131:37949)
I20260810 14:44:29.071419 27505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:38849
I20260810 14:44:29.075012 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 27913
I20260810 14:44:29.075245 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:44:29.075768 25438 external_mini_cluster.cc:1428] Setting key 39a1bd965c2831a9c24e0786b2d09610
I20260810 14:44:29.078831 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:44:29 dist-test-slave-vkjp krb5kdc[27449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 14:44:29 dist-test-slave-vkjp krb5kdc[27449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373069, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:29.131947 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.117068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50580 (local address 127.24.215.190:41921)
0810 14:44:29.117271 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:29.117275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:29.117598 (+ 323us) server_negotiation.cc:407] Connection header received
0810 14:44:29.117779 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:29.117784 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:29.117856 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:29.117976 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:29.119529 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.120307 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.121811 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.122230 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.122335 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.126476 (+ 4141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:29.126482 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:29.126486 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:29.126525 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:29.129334 (+ 2809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:29.130078 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:29.130082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:29.130085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:29.130201 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:29.130577 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:29.130580 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:29.130582 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:29.130858 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:29.131063 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:29.131480 (+ 417us) server_negotiation.cc:299] Negotiation successful
0810 14:44:29.131679 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 14:44:29.136505 27505 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:44:29.143667 27505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:44:29.170737 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.158005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:37949 (local address 127.0.0.1:51416)
0810 14:44:29.159745 (+ 1740us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:29.159767 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:29.160012 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:29.162176 (+ 2164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:29.162187 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:29.162232 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:29.162831 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.162847 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.164983 (+ 2136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.164988 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.167452 (+ 2464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.167469 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.169354 (+ 1885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.169360 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.169570 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.169785 (+ 215us) client_negotiation.cc:769] Sending connection context
0810 14:44:29.169954 (+ 169us) client_negotiation.cc:240] Negotiation successful
0810 14:44:29.170184 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1563,"mutex_wait_us":163,"thread_start_us":1424,"threads_started":1}
I20260810 14:44:29.171176 28102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.158774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51394 (local address 127.24.215.129:45257)
0810 14:44:29.162871 (+ 4097us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:29.162880 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:29.162924 (+ 44us) server_negotiation.cc:407] Connection header received
0810 14:44:29.163019 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:29.163028 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:29.163324 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:29.163524 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:29.164043 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.165523 (+ 1480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.168219 (+ 2696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.169925 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.170186 (+ 261us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.170352 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:29.170648 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 14:44:29.170843 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3921,"thread_start_us":3782,"threads_started":1}
I20260810 14:44:29.171232 28103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.158073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:45257 (local address 127.0.0.1:51394)
0810 14:44:29.160839 (+ 2766us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:29.160855 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:29.160971 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:29.163512 (+ 2541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:29.163516 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:29.163542 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:29.163875 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.163884 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.165674 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.165680 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.168035 (+ 2355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.168047 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.170228 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.170234 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.170452 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.170479 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 14:44:29.170596 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 14:44:29.170745 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2652,"mutex_wait_us":60,"thread_start_us":2522,"threads_started":1}
I20260810 14:44:29.171511 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.157570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56124 (local address 127.24.215.130:36427)
0810 14:44:29.159033 (+ 1463us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:29.159046 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:29.161496 (+ 2450us) server_negotiation.cc:407] Connection header received
0810 14:44:29.161608 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:29.161617 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:29.161897 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:29.162109 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:29.163207 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.165083 (+ 1876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.168010 (+ 2927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.169613 (+ 1603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.169830 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.170005 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:29.170964 (+ 959us) server_negotiation.cc:299] Negotiation successful
0810 14:44:29.171220 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1282,"thread_start_us":1129,"threads_started":1}
I20260810 14:44:29.170818 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.157319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:36427 (local address 127.0.0.1:56124)
0810 14:44:29.158160 (+ 841us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:29.160226 (+ 2066us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:29.160339 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:29.162381 (+ 2042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:29.162389 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:29.162448 (+ 59us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:29.162995 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.163009 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.165219 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.165230 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.167784 (+ 2554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.167800 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.169890 (+ 2090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.169899 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.170107 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.170145 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 14:44:29.170230 (+ 85us) client_negotiation.cc:240] Negotiation successful
0810 14:44:29.170375 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":557,"mutex_wait_us":95,"thread_start_us":381,"threads_started":1}
I20260810 14:44:29.172178 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.158653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51416 (local address 127.24.215.131:37949)
0810 14:44:29.161602 (+ 2949us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:29.161610 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:29.161637 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:44:29.161718 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:29.161724 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:29.161894 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:29.162109 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:29.163056 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.164802 (+ 1746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.167687 (+ 2885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.169120 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.169353 (+ 233us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.169570 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:29.171308 (+ 1738us) server_negotiation.cc:299] Negotiation successful
0810 14:44:29.171527 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2767,"thread_start_us":2580,"threads_started":1}
I20260810 14:44:29.175369 27665 tablet_service.cc:1467] Processing CreateTablet for tablet 87b9c525f373460bb5b04d6c4ecbc168 (DEFAULT_TABLE table=test-table [id=888639c1d789482b8907ff1d1fdda113]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:29.175581 27993 tablet_service.cc:1467] Processing CreateTablet for tablet 87b9c525f373460bb5b04d6c4ecbc168 (DEFAULT_TABLE table=test-table [id=888639c1d789482b8907ff1d1fdda113]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:29.175577 27832 tablet_service.cc:1467] Processing CreateTablet for tablet 87b9c525f373460bb5b04d6c4ecbc168 (DEFAULT_TABLE table=test-table [id=888639c1d789482b8907ff1d1fdda113]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:29.176581 27665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87b9c525f373460bb5b04d6c4ecbc168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:29.176951 27993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87b9c525f373460bb5b04d6c4ecbc168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:29.176959 27832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 87b9c525f373460bb5b04d6c4ecbc168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:29.188865 28104 tablet_bootstrap.cc:492] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff: Bootstrap starting.
I20260810 14:44:29.192971 28104 tablet_bootstrap.cc:654] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:29.194236 28104 log.cc:824] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:29.197420 28104 tablet_bootstrap.cc:492] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff: No bootstrap required, opened a new log
I20260810 14:44:29.197723 28104 ts_tablet_manager.cc:1397] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 14:44:29.199676 28106 tablet_bootstrap.cc:492] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb: Bootstrap starting.
I20260810 14:44:29.199998 28105 tablet_bootstrap.cc:492] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522: Bootstrap starting.
I20260810 14:44:29.203585 28105 tablet_bootstrap.cc:654] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:29.204414 28106 tablet_bootstrap.cc:654] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:29.204836 28105 log.cc:824] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:29.205651 28106 log.cc:824] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:29.208117 28105 tablet_bootstrap.cc:492] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522: No bootstrap required, opened a new log
I20260810 14:44:29.208405 28105 ts_tablet_manager.cc:1397] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 14:44:29.208386 28104 raft_consensus.cc:348] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.208866 28104 raft_consensus.cc:374] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:29.208984 28104 raft_consensus.cc:729] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d013baa9b7d445c8a956138221b53ff, State: Initialized, Role: FOLLOWER
I20260810 14:44:29.209833 28106 tablet_bootstrap.cc:492] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb: No bootstrap required, opened a new log
I20260810 14:44:29.209678 28104 consensus_queue.cc:260] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.210098 28106 ts_tablet_manager.cc:1397] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb: Time spent bootstrapping tablet: real 0.015s user 0.002s sys 0.005s
I20260810 14:44:29.211616 28104 ts_tablet_manager.cc:1428] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
I20260810 14:44:29.218556 28105 raft_consensus.cc:348] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.218935 28105 raft_consensus.cc:374] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:29.219062 28105 raft_consensus.cc:729] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f2682bb26fc42ac8a2f39e943a79522, State: Initialized, Role: FOLLOWER
I20260810 14:44:29.219408 28106 raft_consensus.cc:348] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.219769 28106 raft_consensus.cc:374] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:29.219874 28106 raft_consensus.cc:729] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4fca73b7f7b4e3ca3f5348bbba63fcb, State: Initialized, Role: FOLLOWER
I20260810 14:44:29.219770 28105 consensus_queue.cc:260] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.220410 28106 consensus_queue.cc:260] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.221042 27908 heartbeater.cc:503] Master 127.24.215.190:41921 was elected leader, sending a full tablet report...
I20260810 14:44:29.221499 28078 heartbeater.cc:503] Master 127.24.215.190:41921 was elected leader, sending a full tablet report...
I20260810 14:44:29.221843 28105 ts_tablet_manager.cc:1428] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522: Time spent starting tablet: real 0.013s user 0.005s sys 0.008s
I20260810 14:44:29.222112 28106 ts_tablet_manager.cc:1428] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb: Time spent starting tablet: real 0.012s user 0.011s sys 0.002s
W20260810 14:44:29.222903 27745 tablet.cc:2369] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:44:29.284057 27909 tablet.cc:2369] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:44:29.308362 28112 raft_consensus.cc:482] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:44:29.308771 28112 raft_consensus.cc:504] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.310612 28112 leader_election.cc:288] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d013baa9b7d445c8a956138221b53ff (127.24.215.129:45257), c4fca73b7f7b4e3ca3f5348bbba63fcb (127.24.215.131:37949)
I20260810 14:44:29.320634 28102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.311500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:41731 (local address 127.24.215.129:45257)
0810 14:44:29.311781 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:29.311790 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:29.313026 (+ 1236us) server_negotiation.cc:407] Connection header received
0810 14:44:29.313400 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:29.313408 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:29.313537 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:29.313763 (+ 226us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:29.314702 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.315492 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.318382 (+ 2890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.319597 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.319766 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.319859 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:29.320241 (+ 382us) server_negotiation.cc:299] Negotiation successful
0810 14:44:29.320387 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 14:44:29.320948 28101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.311994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:56085 (local address 127.24.215.131:37949)
0810 14:44:29.312237 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:29.312245 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:29.313287 (+ 1042us) server_negotiation.cc:407] Connection header received
0810 14:44:29.313526 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:29.313532 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:29.313612 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:29.313712 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:29.314497 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.315644 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.318754 (+ 3110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.320002 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.320194 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.320276 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:29.320562 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 14:44:29.320694 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 14:44:29.321143 28116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.311206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:37949 (local address 127.24.215.130:56085)
0810 14:44:29.313178 (+ 1972us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:29.313193 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:29.313380 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:29.313776 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:29.313782 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:29.313827 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:29.314302 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.314316 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.315796 (+ 1480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.315801 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.318571 (+ 2770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.318588 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.320230 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.320237 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.320399 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.320423 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 14:44:29.320534 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 14:44:29.320677 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1835,"mutex_wait_us":58,"thread_start_us":1643,"threads_started":1}
I20260810 14:44:29.321650 28115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.311058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:45257 (local address 127.24.215.130:41731)
0810 14:44:29.312853 (+ 1795us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:29.312879 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:29.313210 (+ 331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:29.314024 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:29.314034 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:29.314055 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:29.314561 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.314578 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.315636 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.315643 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.318198 (+ 2555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.318214 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.319830 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.319837 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.320023 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.320059 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 14:44:29.320823 (+ 764us) client_negotiation.cc:240] Negotiation successful
0810 14:44:29.320977 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1622,"mutex_wait_us":153,"thread_start_us":1376,"threads_started":1}
W20260810 14:44:29.322186 28079 tablet.cc:2369] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:44:29.322270 28018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b9c525f373460bb5b04d6c4ecbc168" candidate_uuid: "1f2682bb26fc42ac8a2f39e943a79522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" is_pre_election: true
I20260810 14:44:29.322875 28018 raft_consensus.cc:2454] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f2682bb26fc42ac8a2f39e943a79522 in term 0.
I20260810 14:44:29.323673 27772 leader_election.cc:302] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f2682bb26fc42ac8a2f39e943a79522, c4fca73b7f7b4e3ca3f5348bbba63fcb; no voters:
I20260810 14:44:29.324118 28112 raft_consensus.cc:2781] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:44:29.324308 28112 raft_consensus.cc:482] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:44:29.324164 27696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b9c525f373460bb5b04d6c4ecbc168" candidate_uuid: "1f2682bb26fc42ac8a2f39e943a79522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d013baa9b7d445c8a956138221b53ff" is_pre_election: true
I20260810 14:44:29.324429 28112 raft_consensus.cc:3037] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:29.324729 27696 raft_consensus.cc:2454] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f2682bb26fc42ac8a2f39e943a79522 in term 0.
I20260810 14:44:29.326867 28112 raft_consensus.cc:504] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.327597 28112 leader_election.cc:288] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [CANDIDATE]: Term 1 election: Requested vote from peers 3d013baa9b7d445c8a956138221b53ff (127.24.215.129:45257), c4fca73b7f7b4e3ca3f5348bbba63fcb (127.24.215.131:37949)
I20260810 14:44:29.328061 27696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b9c525f373460bb5b04d6c4ecbc168" candidate_uuid: "1f2682bb26fc42ac8a2f39e943a79522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d013baa9b7d445c8a956138221b53ff"
I20260810 14:44:29.328233 27696 raft_consensus.cc:3037] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:29.328173 28018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "87b9c525f373460bb5b04d6c4ecbc168" candidate_uuid: "1f2682bb26fc42ac8a2f39e943a79522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb"
I20260810 14:44:29.328383 28018 raft_consensus.cc:3037] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:29.330482 27696 raft_consensus.cc:2454] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f2682bb26fc42ac8a2f39e943a79522 in term 1.
I20260810 14:44:29.330946 27769 leader_election.cc:302] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f2682bb26fc42ac8a2f39e943a79522, 3d013baa9b7d445c8a956138221b53ff; no voters:
I20260810 14:44:29.331228 28018 raft_consensus.cc:2454] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f2682bb26fc42ac8a2f39e943a79522 in term 1.
I20260810 14:44:29.331326 28112 raft_consensus.cc:2781] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:29.331878 28112 raft_consensus.cc:686] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 LEADER]: Becoming Leader. State: Replica: 1f2682bb26fc42ac8a2f39e943a79522, State: Running, Role: LEADER
I20260810 14:44:29.332383 28112 consensus_queue.cc:237] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.337790 27503 catalog_manager.cc:5495] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f2682bb26fc42ac8a2f39e943a79522 (127.24.215.130). New cstate: current_term: 1 leader_uuid: "1f2682bb26fc42ac8a2f39e943a79522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } health_report { overall_health: UNKNOWN } } }
Aug 10 14:44:29 dist-test-slave-vkjp krb5kdc[27449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373067, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:29.395246 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.381706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50582 (local address 127.24.215.190:41921)
0810 14:44:29.381945 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:29.381950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:29.382479 (+ 529us) server_negotiation.cc:407] Connection header received
0810 14:44:29.382623 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:29.382627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:29.382691 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:29.382776 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:29.384020 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.384795 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.386119 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.386523 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.386621 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.389890 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:29.389895 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:29.389898 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:29.389928 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:29.392596 (+ 2668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:29.393387 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:29.393391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:29.393394 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:29.393527 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:29.393869 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:29.393873 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:29.393875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:29.394152 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:29.394286 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:29.394699 (+ 413us) server_negotiation.cc:299] Negotiation successful
0810 14:44:29.394909 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":81}
I20260810 14:44:29.395279 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.381764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41921 (local address 127.0.0.1:50582)
0810 14:44:29.382367 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:29.382383 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:29.382516 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:29.382829 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:29.382832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:29.383594 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:29.383866 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.383875 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.384903 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.384907 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.385954 (+ 1047us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.385967 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.386694 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.386699 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.386812 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.387515 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:29.387533 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:29.389705 (+ 2172us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:29.392748 (+ 3043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:29.392759 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:29.392819 (+ 60us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:29.393269 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:29.393626 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:29.393630 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:29.393632 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:29.393771 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:29.394274 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:29.394280 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:29.394446 (+ 166us) client_negotiation.cc:769] Sending connection context
0810 14:44:29.394589 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 14:44:29.394801 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":354,"threads_started":1}
I20260810 14:44:29.400466 27503 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:50582:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 14:44:29.401942 27503 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:44:29.408865 27665 tablet_service.cc:1467] Processing CreateTablet for tablet dbd4b27afa304c8889b47e6ddbf1503a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=749cd4b80baa4b7fa80fa22567b9fe35]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:44:29.409296 27993 tablet_service.cc:1467] Processing CreateTablet for tablet dbd4b27afa304c8889b47e6ddbf1503a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=749cd4b80baa4b7fa80fa22567b9fe35]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:44:29.409919 27665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd4b27afa304c8889b47e6ddbf1503a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:29.410135 27993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd4b27afa304c8889b47e6ddbf1503a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:29.412761 27832 tablet_service.cc:1467] Processing CreateTablet for tablet dbd4b27afa304c8889b47e6ddbf1503a (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=749cd4b80baa4b7fa80fa22567b9fe35]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:44:29.413794 27832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dbd4b27afa304c8889b47e6ddbf1503a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:29.420859 28106 tablet_bootstrap.cc:492] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb: Bootstrap starting.
I20260810 14:44:29.422232 28106 tablet_bootstrap.cc:654] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:29.424728 28106 tablet_bootstrap.cc:492] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb: No bootstrap required, opened a new log
I20260810 14:44:29.424865 28106 ts_tablet_manager.cc:1397] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.001s
I20260810 14:44:29.425161 28104 tablet_bootstrap.cc:492] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff: Bootstrap starting.
I20260810 14:44:29.425694 28106 raft_consensus.cc:348] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.425976 28106 raft_consensus.cc:374] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:29.426088 28106 raft_consensus.cc:729] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4fca73b7f7b4e3ca3f5348bbba63fcb, State: Initialized, Role: FOLLOWER
I20260810 14:44:29.426386 28106 consensus_queue.cc:260] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.426890 28113 tablet_replica.cc:439] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } } }
I20260810 14:44:29.427299 28113 tablet_replica.cc:442] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:44:29.427764 28104 tablet_bootstrap.cc:654] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:29.427804 28106 ts_tablet_manager.cc:1428] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 14:44:29.428737 28105 tablet_bootstrap.cc:492] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522: Bootstrap starting.
I20260810 14:44:29.431069 28105 tablet_bootstrap.cc:654] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:29.432610 28104 tablet_bootstrap.cc:492] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff: No bootstrap required, opened a new log
I20260810 14:44:29.432783 28104 ts_tablet_manager.cc:1397] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 14:44:29.433314 28105 tablet_bootstrap.cc:492] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522: No bootstrap required, opened a new log
I20260810 14:44:29.433471 28105 ts_tablet_manager.cc:1397] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 14:44:29.433642 28104 raft_consensus.cc:348] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.433903 28104 raft_consensus.cc:374] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:29.434007 28104 raft_consensus.cc:729] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d013baa9b7d445c8a956138221b53ff, State: Initialized, Role: FOLLOWER
I20260810 14:44:29.434259 28104 consensus_queue.cc:260] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.434300 28105 raft_consensus.cc:348] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.434557 28105 raft_consensus.cc:374] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:29.434652 28105 raft_consensus.cc:729] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f2682bb26fc42ac8a2f39e943a79522, State: Initialized, Role: FOLLOWER
I20260810 14:44:29.434890 28105 consensus_queue.cc:260] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.434805 28111 tablet_replica.cc:439] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } } }
I20260810 14:44:29.435324 28111 tablet_replica.cc:442] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:44:29.435324 28112 tablet_replica.cc:439] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } } }
I20260810 14:44:29.435600 28112 tablet_replica.cc:442] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:44:29.436098 28105 ts_tablet_manager.cc:1428] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:44:29.437302 28104 ts_tablet_manager.cc:1428] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260810 14:44:29.443243 28112 raft_consensus.cc:482] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:44:29.443463 28112 raft_consensus.cc:504] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.444222 28112 leader_election.cc:288] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d013baa9b7d445c8a956138221b53ff (127.24.215.129:45257), c4fca73b7f7b4e3ca3f5348bbba63fcb (127.24.215.131:37949)
I20260810 14:44:29.444705 27696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd4b27afa304c8889b47e6ddbf1503a" candidate_uuid: "1f2682bb26fc42ac8a2f39e943a79522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d013baa9b7d445c8a956138221b53ff" is_pre_election: true
I20260810 14:44:29.445005 27696 raft_consensus.cc:2454] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f2682bb26fc42ac8a2f39e943a79522 in term 0.
I20260810 14:44:29.444970 28018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd4b27afa304c8889b47e6ddbf1503a" candidate_uuid: "1f2682bb26fc42ac8a2f39e943a79522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" is_pre_election: true
I20260810 14:44:29.445222 28018 raft_consensus.cc:2454] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f2682bb26fc42ac8a2f39e943a79522 in term 0.
I20260810 14:44:29.445425 27769 leader_election.cc:302] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f2682bb26fc42ac8a2f39e943a79522, 3d013baa9b7d445c8a956138221b53ff; no voters:
I20260810 14:44:29.445875 28112 raft_consensus.cc:2781] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:44:29.445987 28112 raft_consensus.cc:482] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:44:29.446134 28112 raft_consensus.cc:3037] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:29.448277 28112 raft_consensus.cc:504] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.448833 28112 leader_election.cc:288] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [CANDIDATE]: Term 1 election: Requested vote from peers 3d013baa9b7d445c8a956138221b53ff (127.24.215.129:45257), c4fca73b7f7b4e3ca3f5348bbba63fcb (127.24.215.131:37949)
I20260810 14:44:29.449291 28018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd4b27afa304c8889b47e6ddbf1503a" candidate_uuid: "1f2682bb26fc42ac8a2f39e943a79522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb"
I20260810 14:44:29.449301 27696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dbd4b27afa304c8889b47e6ddbf1503a" candidate_uuid: "1f2682bb26fc42ac8a2f39e943a79522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d013baa9b7d445c8a956138221b53ff"
I20260810 14:44:29.449518 27696 raft_consensus.cc:3037] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:29.449513 28018 raft_consensus.cc:3037] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:29.451709 28018 raft_consensus.cc:2454] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f2682bb26fc42ac8a2f39e943a79522 in term 1.
I20260810 14:44:29.451709 27696 raft_consensus.cc:2454] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f2682bb26fc42ac8a2f39e943a79522 in term 1.
I20260810 14:44:29.452198 27769 leader_election.cc:302] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f2682bb26fc42ac8a2f39e943a79522, 3d013baa9b7d445c8a956138221b53ff; no voters:
I20260810 14:44:29.452520 28112 raft_consensus.cc:2781] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:29.452706 28112 raft_consensus.cc:686] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 LEADER]: Becoming Leader. State: Replica: 1f2682bb26fc42ac8a2f39e943a79522, State: Running, Role: LEADER
I20260810 14:44:29.452976 28112 consensus_queue.cc:237] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.454651 28118 tablet_replica.cc:439] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522: TxnStatusTablet state changed. Reason: New leader 1f2682bb26fc42ac8a2f39e943a79522. Latest consensus state: current_term: 1 leader_uuid: "1f2682bb26fc42ac8a2f39e943a79522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } } }
I20260810 14:44:29.455019 28118 tablet_replica.cc:442] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:44:29.455838 28133 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 14:44:29.456122 28133 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 14:44:29.456178 27503 catalog_manager.cc:5495] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f2682bb26fc42ac8a2f39e943a79522 (127.24.215.130). New cstate: current_term: 1 leader_uuid: "1f2682bb26fc42ac8a2f39e943a79522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } health_report { overall_health: UNKNOWN } } }
I20260810 14:44:29.479736 28127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.472905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:36427 (local address 127.0.0.1:56132)
0810 14:44:29.473210 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:29.473236 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:29.473392 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:29.474166 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:29.474170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:29.474181 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:44:29.474345 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.474352 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.475840 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.475844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.476885 (+ 1041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.476898 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.477889 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.477893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.478060 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.478119 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:44:29.479209 (+ 1090us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:44:29.479238 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 14:44:29.479354 (+ 116us) client_negotiation.cc:240] Negotiation successful
0810 14:44:29.479481 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260810 14:44:29.479770 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.473065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56132 (local address 127.24.215.130:36427)
0810 14:44:29.473711 (+ 646us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:29.473717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:29.473735 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:44:29.473856 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:29.473862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:29.473961 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:29.474122 (+ 161us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:29.474497 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.475701 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.477040 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.477713 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.477850 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.478331 (+ 481us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:29.479050 (+ 719us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:29.479237 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:29.479383 (+ 146us) server_negotiation.cc:299] Negotiation successful
0810 14:44:29.479483 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508}
I20260810 14:44:29.487696 27696 raft_consensus.cc:1261] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Refusing update from remote peer 1f2682bb26fc42ac8a2f39e943a79522: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:29.487696 28018 raft_consensus.cc:1261] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Refusing update from remote peer 1f2682bb26fc42ac8a2f39e943a79522: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:29.488757 28118 consensus_queue.cc:1035] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:29.489113 28112 consensus_queue.cc:1035] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:29.497797 28113 tablet_replica.cc:439] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb: TxnStatusTablet state changed. Reason: New leader 1f2682bb26fc42ac8a2f39e943a79522. Latest consensus state: current_term: 1 leader_uuid: "1f2682bb26fc42ac8a2f39e943a79522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } } }
I20260810 14:44:29.498122 28113 tablet_replica.cc:442] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:44:29.501078 28137 mvcc.cc:205] Tried to move back new op lower bound from 7316984092611100672 to 7316984092481863680. Current Snapshot: MvccSnapshot[applied={T|T < 7316984092611100672}]
I20260810 14:44:29.502076 28118 tablet_replica.cc:439] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f2682bb26fc42ac8a2f39e943a79522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } } }
I20260810 14:44:29.502369 28118 tablet_replica.cc:442] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:44:29.502425 28138 mvcc.cc:205] Tried to move back new op lower bound from 7316984092611100672 to 7316984092481863680. Current Snapshot: MvccSnapshot[applied={T|T < 7316984092611100672}]
I20260810 14:44:29.507290 28113 tablet_replica.cc:439] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f2682bb26fc42ac8a2f39e943a79522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } } }
I20260810 14:44:29.507562 28113 tablet_replica.cc:442] T dbd4b27afa304c8889b47e6ddbf1503a P c4fca73b7f7b4e3ca3f5348bbba63fcb: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:44:29.513679 28111 tablet_replica.cc:439] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff: TxnStatusTablet state changed. Reason: New leader 1f2682bb26fc42ac8a2f39e943a79522. Latest consensus state: current_term: 1 leader_uuid: "1f2682bb26fc42ac8a2f39e943a79522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } } }
I20260810 14:44:29.514039 28111 tablet_replica.cc:442] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:44:29.516767 28112 tablet_replica.cc:439] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1f2682bb26fc42ac8a2f39e943a79522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } } }
I20260810 14:44:29.517078 28112 tablet_replica.cc:442] T dbd4b27afa304c8889b47e6ddbf1503a P 1f2682bb26fc42ac8a2f39e943a79522: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:44:29.520658 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.513285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56136 (local address 127.24.215.130:36427)
0810 14:44:29.513475 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:29.513482 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:29.513500 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:44:29.513563 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:29.513568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:29.513687 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:29.513781 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:29.514409 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.515550 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.517509 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.518156 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.518280 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.518951 (+ 671us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:29.519191 (+ 240us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:29.519294 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:29.520055 (+ 761us) server_negotiation.cc:299] Negotiation successful
0810 14:44:29.520173 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 14:44:29.520920 28111 tablet_replica.cc:439] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1f2682bb26fc42ac8a2f39e943a79522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } } }
I20260810 14:44:29.521184 28111 tablet_replica.cc:442] T dbd4b27afa304c8889b47e6ddbf1503a P 3d013baa9b7d445c8a956138221b53ff: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:44:29.547353 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.533008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:52207 (local address 127.24.215.130:36427)
0810 14:44:29.533199 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:29.533205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:29.539867 (+ 6662us) server_negotiation.cc:407] Connection header received
0810 14:44:29.540073 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:29.540078 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:29.540175 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:29.540278 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:29.540961 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.542168 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.544913 (+ 2745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.545684 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.545826 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.546484 (+ 658us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:29.546639 (+ 155us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:29.546734 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:29.546897 (+ 163us) server_negotiation.cc:299] Negotiation successful
0810 14:44:29.547013 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":52}
I20260810 14:44:29.547758 28149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:29.532927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:36427 (local address 127.24.215.130:52207)
0810 14:44:29.539708 (+ 6781us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:29.539732 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:29.539920 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:29.540316 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:29.540321 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:29.540334 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:44:29.540785 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.540802 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.542659 (+ 1857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.542666 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.544747 (+ 2081us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:29.544764 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:29.545913 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:29.545920 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:29.546115 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:29.546211 (+ 96us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:44:29.546756 (+ 545us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:44:29.546770 (+ 14us) client_negotiation.cc:769] Sending connection context
0810 14:44:29.546859 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 14:44:29.547041 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6579,"mutex_wait_us":1,"thread_start_us":6370,"threads_started":1}
I20260810 14:44:29.561048 27696 raft_consensus.cc:1261] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Refusing update from remote peer 1f2682bb26fc42ac8a2f39e943a79522: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:29.561980 28018 raft_consensus.cc:1261] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Refusing update from remote peer 1f2682bb26fc42ac8a2f39e943a79522: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:29.562589 28112 consensus_queue.cc:1035] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:29.563133 28112 consensus_queue.cc:1035] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:29.637375 27503 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:44:29.637666 27503 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:44:29.641474 27503 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d: Sending DeleteTablet for 3 replicas of tablet 87b9c525f373460bb5b04d6c4ecbc168
I20260810 14:44:29.642836 27993 tablet_service.cc:1514] Processing DeleteTablet for tablet 87b9c525f373460bb5b04d6c4ecbc168 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:29 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:51416
I20260810 14:44:29.642962 27665 tablet_service.cc:1514] Processing DeleteTablet for tablet 87b9c525f373460bb5b04d6c4ecbc168 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:29 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:51394
I20260810 14:44:29.642973 27832 tablet_service.cc:1514] Processing DeleteTablet for tablet 87b9c525f373460bb5b04d6c4ecbc168 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:29 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:56124
I20260810 14:44:29.645220 27503 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50580:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:44:29.646543 27503 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:44:29.654938 28159 tablet_replica.cc:330] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522: stopping tablet replica
I20260810 14:44:29.655445 28159 raft_consensus.cc:2229] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:44:29.656262 28159 raft_consensus.cc:2258] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:29.658655 28159 ts_tablet_manager.cc:1905] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:29.661999 28158 tablet_replica.cc:330] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb: stopping tablet replica
I20260810 14:44:29.661988 27664 tablet_service.cc:1467] Processing CreateTablet for tablet 632ce366df9c45ce9a88c467eb7a573a (DEFAULT_TABLE table=test-table [id=cf582ea197b1434da629a3cbec434984]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:29.662523 28158 raft_consensus.cc:2229] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:44:29.662672 27664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 632ce366df9c45ce9a88c467eb7a573a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:29.662926 28158 raft_consensus.cc:2258] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:29.666814 27992 tablet_service.cc:1467] Processing CreateTablet for tablet 632ce366df9c45ce9a88c467eb7a573a (DEFAULT_TABLE table=test-table [id=cf582ea197b1434da629a3cbec434984]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:29.666930 27832 tablet_service.cc:1467] Processing CreateTablet for tablet 632ce366df9c45ce9a88c467eb7a573a (DEFAULT_TABLE table=test-table [id=cf582ea197b1434da629a3cbec434984]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:29.667310 27992 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 632ce366df9c45ce9a88c467eb7a573a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:29.667670 27832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 632ce366df9c45ce9a88c467eb7a573a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:29.668469 28158 ts_tablet_manager.cc:1905] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:29.673357 28157 tablet_replica.cc:330] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff: stopping tablet replica
I20260810 14:44:29.673885 28157 raft_consensus.cc:2229] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:44:29.674342 28106 tablet_bootstrap.cc:492] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb: Bootstrap starting.
I20260810 14:44:29.675223 28157 raft_consensus.cc:2258] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:29.676083 28106 tablet_bootstrap.cc:654] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:29.681502 28157 ts_tablet_manager.cc:1905] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:29.681527 28159 ts_tablet_manager.cc:1918] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:44:29.681695 28159 log.cc:1196] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/87b9c525f373460bb5b04d6c4ecbc168
I20260810 14:44:29.682307 28159 ts_tablet_manager.cc:1939] T 87b9c525f373460bb5b04d6c4ecbc168 P 1f2682bb26fc42ac8a2f39e943a79522: Deleting consensus metadata
I20260810 14:44:29.684382 27484 catalog_manager.cc:4841] TS 1f2682bb26fc42ac8a2f39e943a79522 (127.24.215.130:36427): tablet 87b9c525f373460bb5b04d6c4ecbc168 (table test-table [id=888639c1d789482b8907ff1d1fdda113]) successfully deleted
I20260810 14:44:29.687894 28158 ts_tablet_manager.cc:1918] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:44:29.688099 28158 log.cc:1196] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/87b9c525f373460bb5b04d6c4ecbc168
I20260810 14:44:29.688717 28158 ts_tablet_manager.cc:1939] T 87b9c525f373460bb5b04d6c4ecbc168 P c4fca73b7f7b4e3ca3f5348bbba63fcb: Deleting consensus metadata
I20260810 14:44:29.688834 28105 tablet_bootstrap.cc:492] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522: Bootstrap starting.
I20260810 14:44:29.690572 28105 tablet_bootstrap.cc:654] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:29.690580 27486 catalog_manager.cc:4841] TS c4fca73b7f7b4e3ca3f5348bbba63fcb (127.24.215.131:37949): tablet 87b9c525f373460bb5b04d6c4ecbc168 (table test-table [id=888639c1d789482b8907ff1d1fdda113]) successfully deleted
I20260810 14:44:29.691620 28104 tablet_bootstrap.cc:492] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff: Bootstrap starting.
I20260810 14:44:29.692355 28106 tablet_bootstrap.cc:492] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb: No bootstrap required, opened a new log
I20260810 14:44:29.692507 28106 ts_tablet_manager.cc:1397] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.003s
I20260810 14:44:29.693254 28106 raft_consensus.cc:348] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.693497 28106 raft_consensus.cc:374] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:29.693568 28106 raft_consensus.cc:729] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4fca73b7f7b4e3ca3f5348bbba63fcb, State: Initialized, Role: FOLLOWER
I20260810 14:44:29.693889 28104 tablet_bootstrap.cc:654] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:29.695205 28106 consensus_queue.cc:260] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.697494 28157 ts_tablet_manager.cc:1918] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:44:29.697711 28157 log.cc:1196] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/87b9c525f373460bb5b04d6c4ecbc168
I20260810 14:44:29.698344 28106 ts_tablet_manager.cc:1428] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb: Time spent starting tablet: real 0.006s user 0.002s sys 0.000s
I20260810 14:44:29.698431 28157 ts_tablet_manager.cc:1939] T 87b9c525f373460bb5b04d6c4ecbc168 P 3d013baa9b7d445c8a956138221b53ff: Deleting consensus metadata
I20260810 14:44:29.701287 27481 catalog_manager.cc:4841] TS 3d013baa9b7d445c8a956138221b53ff (127.24.215.129:45257): tablet 87b9c525f373460bb5b04d6c4ecbc168 (table test-table [id=888639c1d789482b8907ff1d1fdda113]) successfully deleted
I20260810 14:44:29.704109 28104 tablet_bootstrap.cc:492] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff: No bootstrap required, opened a new log
I20260810 14:44:29.704348 28104 ts_tablet_manager.cc:1397] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.009s
I20260810 14:44:29.705052 28104 raft_consensus.cc:348] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.705322 28104 raft_consensus.cc:374] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:29.705451 28104 raft_consensus.cc:729] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d013baa9b7d445c8a956138221b53ff, State: Initialized, Role: FOLLOWER
I20260810 14:44:29.705791 28104 consensus_queue.cc:260] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.706457 28104 ts_tablet_manager.cc:1428] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260810 14:44:29.711112 28105 tablet_bootstrap.cc:492] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522: No bootstrap required, opened a new log
I20260810 14:44:29.711318 28105 ts_tablet_manager.cc:1397] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522: Time spent bootstrapping tablet: real 0.023s user 0.007s sys 0.004s
I20260810 14:44:29.712137 28105 raft_consensus.cc:348] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.712419 28105 raft_consensus.cc:374] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:29.712505 28105 raft_consensus.cc:729] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f2682bb26fc42ac8a2f39e943a79522, State: Initialized, Role: FOLLOWER
I20260810 14:44:29.712754 28105 consensus_queue.cc:260] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:29.713605 28105 ts_tablet_manager.cc:1428] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 14:44:30.005071 28113 raft_consensus.cc:482] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:44:30.005376 28113 raft_consensus.cc:504] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:30.007643 28113 leader_election.cc:288] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d013baa9b7d445c8a956138221b53ff (127.24.215.129:45257), 1f2682bb26fc42ac8a2f39e943a79522 (127.24.215.130:36427)
I20260810 14:44:30.016839 28165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:30.007909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:45257 (local address 127.24.215.131:45145)
0810 14:44:30.009714 (+ 1805us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:30.009729 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:30.009914 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:30.010252 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:30.010257 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:30.010281 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:30.010667 (+ 386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:30.010684 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.012396 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.012401 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:30.014689 (+ 2288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:30.014701 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.015901 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.015907 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:30.016125 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:30.016165 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 14:44:30.016266 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 14:44:30.016400 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1703,"mutex_wait_us":836,"thread_start_us":1057,"threads_started":1}
I20260810 14:44:30.017030 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:30.008152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:43127 (local address 127.24.215.130:36427)
0810 14:44:30.008403 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:30.008412 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:30.009096 (+ 684us) server_negotiation.cc:407] Connection header received
0810 14:44:30.009513 (+ 417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:30.009523 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:30.009618 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:30.009705 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:30.010614 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.011536 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.014316 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.016001 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.016189 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:30.016309 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:30.016589 (+ 280us) server_negotiation.cc:299] Negotiation successful
0810 14:44:30.016728 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 14:44:30.017483 28164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:30.008203 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:45145 (local address 127.24.215.129:45257)
0810 14:44:30.009259 (+ 1056us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:30.009269 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:30.009843 (+ 574us) server_negotiation.cc:407] Connection header received
0810 14:44:30.010063 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:30.010066 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:30.010123 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:30.010233 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:30.010873 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.012194 (+ 1321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.014836 (+ 2642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.015729 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.015903 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:30.015984 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:30.016421 (+ 437us) server_negotiation.cc:299] Negotiation successful
0810 14:44:30.016593 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":892,"thread_start_us":781,"threads_started":1}
I20260810 14:44:30.017683 28163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:30.007906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:36427 (local address 127.24.215.131:43127)
0810 14:44:30.008961 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:30.008985 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:30.009260 (+ 275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:30.009984 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:30.009990 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:30.010024 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:30.010383 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:30.010404 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.011691 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.011697 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:30.014100 (+ 2403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:30.014116 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.016194 (+ 2078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.016200 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:30.016378 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:30.016405 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 14:44:30.017215 (+ 810us) client_negotiation.cc:240] Negotiation successful
0810 14:44:30.017367 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920,"thread_start_us":736,"threads_started":1}
I20260810 14:44:30.018250 27696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "632ce366df9c45ce9a88c467eb7a573a" candidate_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d013baa9b7d445c8a956138221b53ff" is_pre_election: true
I20260810 14:44:30.018565 27696 raft_consensus.cc:2454] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4fca73b7f7b4e3ca3f5348bbba63fcb in term 0.
I20260810 14:44:30.018608 27858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "632ce366df9c45ce9a88c467eb7a573a" candidate_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f2682bb26fc42ac8a2f39e943a79522" is_pre_election: true
I20260810 14:44:30.019012 27858 raft_consensus.cc:2454] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c4fca73b7f7b4e3ca3f5348bbba63fcb in term 0.
I20260810 14:44:30.019224 27936 leader_election.cc:302] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d013baa9b7d445c8a956138221b53ff, c4fca73b7f7b4e3ca3f5348bbba63fcb; no voters:
I20260810 14:44:30.019712 28113 raft_consensus.cc:2781] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:44:30.019860 28113 raft_consensus.cc:482] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:44:30.019990 28113 raft_consensus.cc:3037] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:30.022276 28113 raft_consensus.cc:504] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:30.022876 28113 leader_election.cc:288] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [CANDIDATE]: Term 1 election: Requested vote from peers 3d013baa9b7d445c8a956138221b53ff (127.24.215.129:45257), 1f2682bb26fc42ac8a2f39e943a79522 (127.24.215.130:36427)
I20260810 14:44:30.023394 27696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "632ce366df9c45ce9a88c467eb7a573a" candidate_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d013baa9b7d445c8a956138221b53ff"
I20260810 14:44:30.023455 27858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "632ce366df9c45ce9a88c467eb7a573a" candidate_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f2682bb26fc42ac8a2f39e943a79522"
I20260810 14:44:30.023620 27696 raft_consensus.cc:3037] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:30.023679 27858 raft_consensus.cc:3037] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:30.025856 27858 raft_consensus.cc:2454] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4fca73b7f7b4e3ca3f5348bbba63fcb in term 1.
I20260810 14:44:30.026286 27938 leader_election.cc:302] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f2682bb26fc42ac8a2f39e943a79522, c4fca73b7f7b4e3ca3f5348bbba63fcb; no voters:
I20260810 14:44:30.026592 28113 raft_consensus.cc:2781] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:30.026695 27696 raft_consensus.cc:2454] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c4fca73b7f7b4e3ca3f5348bbba63fcb in term 1.
I20260810 14:44:30.027176 28113 raft_consensus.cc:686] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 LEADER]: Becoming Leader. State: Replica: c4fca73b7f7b4e3ca3f5348bbba63fcb, State: Running, Role: LEADER
I20260810 14:44:30.027595 28113 consensus_queue.cc:237] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:30.031348 27503 catalog_manager.cc:5495] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb reported cstate change: term changed from 0 to 1, leader changed from <none> to c4fca73b7f7b4e3ca3f5348bbba63fcb (127.24.215.131). New cstate: current_term: 1 leader_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } health_report { overall_health: HEALTHY } } }
I20260810 14:44:30.092478 28170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:30.068503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51426 (local address 127.24.215.131:37949)
0810 14:44:30.084249 (+ 15746us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:30.084256 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:30.084293 (+ 37us) server_negotiation.cc:407] Connection header received
0810 14:44:30.084412 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:30.084418 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:30.084530 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:30.084721 (+ 191us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:30.085539 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.086972 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.089213 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.089904 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.090041 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:30.090659 (+ 618us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:30.091542 (+ 883us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:30.091731 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:30.091874 (+ 143us) server_negotiation.cc:299] Negotiation successful
0810 14:44:30.091976 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15579,"thread_start_us":15400,"threads_started":1}
I20260810 14:44:30.110575 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:30.104693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:50573 (local address 127.24.215.130:36427)
0810 14:44:30.104948 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:30.104956 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:30.105304 (+ 348us) server_negotiation.cc:407] Connection header received
0810 14:44:30.105533 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:30.105539 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:30.105644 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:30.105754 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:30.106306 (+ 552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.107065 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.108538 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.109020 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.109191 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:30.109663 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:30.109794 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:30.109921 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:30.110069 (+ 148us) server_negotiation.cc:299] Negotiation successful
0810 14:44:30.110186 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 14:44:30.110631 28173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:30.104521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:36427 (local address 127.24.215.131:50573)
0810 14:44:30.105179 (+ 658us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:30.105196 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:30.105376 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:30.105779 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:30.105785 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:30.105799 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:44:30.106135 (+ 336us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:30.106144 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.107250 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.107257 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:30.108364 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:30.108376 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.109231 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.109239 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:30.109387 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:30.109442 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:44:30.109916 (+ 474us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:44:30.109932 (+ 16us) client_negotiation.cc:769] Sending connection context
0810 14:44:30.110044 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 14:44:30.110199 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"thread_start_us":301,"threads_started":1}
I20260810 14:44:30.120276 27696 raft_consensus.cc:1261] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Refusing update from remote peer c4fca73b7f7b4e3ca3f5348bbba63fcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:30.120865 27858 raft_consensus.cc:1261] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 FOLLOWER]: Refusing update from remote peer c4fca73b7f7b4e3ca3f5348bbba63fcb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:30.120983 28113 consensus_queue.cc:1035] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:30.121661 28113 consensus_queue.cc:1035] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:30.165974 28170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:30.147560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:33117 (local address 127.24.215.131:37949)
0810 14:44:30.147732 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:30.147739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:30.159806 (+ 12067us) server_negotiation.cc:407] Connection header received
0810 14:44:30.159939 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:30.159945 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:30.160066 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:30.160237 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:30.160746 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.161882 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.163639 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.164332 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.164485 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:30.165238 (+ 753us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:30.165358 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:30.165459 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:30.165588 (+ 129us) server_negotiation.cc:299] Negotiation successful
0810 14:44:30.165670 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 14:44:30.166198 28178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:30.147509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:37949 (local address 127.24.215.130:33117)
0810 14:44:30.159675 (+ 12166us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:30.159694 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:30.159821 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:30.160253 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:30.160257 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:30.160267 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:44:30.160565 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:30.160580 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.162041 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.162045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:30.163468 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:30.163482 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:30.164772 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:30.164785 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:30.164981 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:30.165042 (+ 61us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:44:30.165470 (+ 428us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:44:30.165485 (+ 15us) client_negotiation.cc:769] Sending connection context
0810 14:44:30.165567 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 14:44:30.165719 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12019,"thread_start_us":12388,"threads_started":1}
W20260810 14:44:30.167511 28138 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 14:44:31.786559 27505 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:44:31.786815 27505 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50580:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:44:31.791353 27505 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d: Sending DeleteTablet for 3 replicas of tablet 632ce366df9c45ce9a88c467eb7a573a
I20260810 14:44:31.792182 27664 tablet_service.cc:1514] Processing DeleteTablet for tablet 632ce366df9c45ce9a88c467eb7a573a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:31 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:51394
I20260810 14:44:31.792243 27832 tablet_service.cc:1514] Processing DeleteTablet for tablet 632ce366df9c45ce9a88c467eb7a573a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:31 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:56124
I20260810 14:44:31.792532 27993 tablet_service.cc:1514] Processing DeleteTablet for tablet 632ce366df9c45ce9a88c467eb7a573a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:31 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:51416
I20260810 14:44:31.804994 28214 tablet_replica.cc:330] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff: stopping tablet replica
I20260810 14:44:31.805429 28214 raft_consensus.cc:2229] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:44:31.805887 28214 raft_consensus.cc:2258] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:31.807410 28214 ts_tablet_manager.cc:1905] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:31.815608 28215 tablet_replica.cc:330] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb: stopping tablet replica
I20260810 14:44:31.815848 28213 tablet_replica.cc:330] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522: stopping tablet replica
I20260810 14:44:31.815862 28215 raft_consensus.cc:2229] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:44:31.816063 28213 raft_consensus.cc:2229] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:44:31.816362 28213 raft_consensus.cc:2258] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:31.816531 28215 raft_consensus.cc:2258] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:31.816828 28214 ts_tablet_manager.cc:1918] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:44:31.816970 28214 log.cc:1196] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/632ce366df9c45ce9a88c467eb7a573a
I20260810 14:44:31.817402 28214 ts_tablet_manager.cc:1939] T 632ce366df9c45ce9a88c467eb7a573a P 3d013baa9b7d445c8a956138221b53ff: Deleting consensus metadata
I20260810 14:44:31.817647 28215 ts_tablet_manager.cc:1905] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:31.818620 27481 catalog_manager.cc:4841] TS 3d013baa9b7d445c8a956138221b53ff (127.24.215.129:45257): tablet 632ce366df9c45ce9a88c467eb7a573a (table test-table [id=cf582ea197b1434da629a3cbec434984]) successfully deleted
I20260810 14:44:31.819003 28213 ts_tablet_manager.cc:1905] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:31.824498 28215 ts_tablet_manager.cc:1918] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:44:31.824740 28215 log.cc:1196] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/632ce366df9c45ce9a88c467eb7a573a
I20260810 14:44:31.825198 28215 ts_tablet_manager.cc:1939] T 632ce366df9c45ce9a88c467eb7a573a P c4fca73b7f7b4e3ca3f5348bbba63fcb: Deleting consensus metadata
I20260810 14:44:31.825490 28213 ts_tablet_manager.cc:1918] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:44:31.825646 28213 log.cc:1196] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/632ce366df9c45ce9a88c467eb7a573a
I20260810 14:44:31.826251 28213 ts_tablet_manager.cc:1939] T 632ce366df9c45ce9a88c467eb7a573a P 1f2682bb26fc42ac8a2f39e943a79522: Deleting consensus metadata
I20260810 14:44:31.826354 27486 catalog_manager.cc:4841] TS c4fca73b7f7b4e3ca3f5348bbba63fcb (127.24.215.131:37949): tablet 632ce366df9c45ce9a88c467eb7a573a (table test-table [id=cf582ea197b1434da629a3cbec434984]) successfully deleted
I20260810 14:44:31.827528 27484 catalog_manager.cc:4841] TS 1f2682bb26fc42ac8a2f39e943a79522 (127.24.215.130:36427): tablet 632ce366df9c45ce9a88c467eb7a573a (table test-table [id=cf582ea197b1434da629a3cbec434984]) successfully deleted
Aug 10 14:44:31 dist-test-slave-vkjp krb5kdc[27449](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373071, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 14:44:31 dist-test-slave-vkjp krb5kdc[27449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373071, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:31.863386 28227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:31.847042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56108 (local address 127.24.215.190:41921)
0810 14:44:31.847794 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:31.847799 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:31.847877 (+ 78us) server_negotiation.cc:407] Connection header received
0810 14:44:31.848038 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:31.848043 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:31.848109 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:31.848207 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:31.849701 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:31.850482 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:31.852441 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:31.852848 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:31.853006 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:31.857650 (+ 4644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:31.857656 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:31.857659 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:31.857694 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:31.860659 (+ 2965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:31.861447 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:31.861452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:31.861455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:31.861673 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:31.862005 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:31.862008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:31.862010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:31.862385 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:31.862575 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:31.862828 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 14:44:31.863009 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":635,"thread_start_us":423,"threads_started":1}
I20260810 14:44:31.867272 27505 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:44:31.867950 27505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:44:31.874869 27664 tablet_service.cc:1467] Processing CreateTablet for tablet 933a8b4a05d849ff82318a743304e0b2 (DEFAULT_TABLE table=test-table [id=f607cbce35774c81aa9e2d30527d8129]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:31.875406 27664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 933a8b4a05d849ff82318a743304e0b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:31.875351 27993 tablet_service.cc:1467] Processing CreateTablet for tablet 933a8b4a05d849ff82318a743304e0b2 (DEFAULT_TABLE table=test-table [id=f607cbce35774c81aa9e2d30527d8129]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:31.875759 27993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 933a8b4a05d849ff82318a743304e0b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:31.877573 27832 tablet_service.cc:1467] Processing CreateTablet for tablet 933a8b4a05d849ff82318a743304e0b2 (DEFAULT_TABLE table=test-table [id=f607cbce35774c81aa9e2d30527d8129]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:31.878227 27832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 933a8b4a05d849ff82318a743304e0b2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:31.881841 28228 tablet_bootstrap.cc:492] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb: Bootstrap starting.
I20260810 14:44:31.882512 28229 tablet_bootstrap.cc:492] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff: Bootstrap starting.
I20260810 14:44:31.884265 28228 tablet_bootstrap.cc:654] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:31.884757 28229 tablet_bootstrap.cc:654] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:31.885334 28230 tablet_bootstrap.cc:492] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522: Bootstrap starting.
I20260810 14:44:31.887482 28228 tablet_bootstrap.cc:492] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb: No bootstrap required, opened a new log
I20260810 14:44:31.887656 28228 ts_tablet_manager.cc:1397] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260810 14:44:31.888211 28230 tablet_bootstrap.cc:654] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:31.888276 28229 tablet_bootstrap.cc:492] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff: No bootstrap required, opened a new log
I20260810 14:44:31.888411 28229 ts_tablet_manager.cc:1397] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260810 14:44:31.888692 28228 raft_consensus.cc:348] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:31.888993 28228 raft_consensus.cc:374] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:31.889272 28228 raft_consensus.cc:729] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4fca73b7f7b4e3ca3f5348bbba63fcb, State: Initialized, Role: FOLLOWER
I20260810 14:44:31.889565 28228 consensus_queue.cc:260] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:31.889591 28229 raft_consensus.cc:348] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:31.890026 28229 raft_consensus.cc:374] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:31.890178 28229 raft_consensus.cc:729] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d013baa9b7d445c8a956138221b53ff, State: Initialized, Role: FOLLOWER
I20260810 14:44:31.890501 28229 consensus_queue.cc:260] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:31.890962 28228 ts_tablet_manager.cc:1428] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260810 14:44:31.891566 28229 ts_tablet_manager.cc:1428] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260810 14:44:31.892449 28230 tablet_bootstrap.cc:492] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522: No bootstrap required, opened a new log
I20260810 14:44:31.892642 28230 ts_tablet_manager.cc:1397] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260810 14:44:31.893707 28230 raft_consensus.cc:348] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:31.894047 28230 raft_consensus.cc:374] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:31.894161 28230 raft_consensus.cc:729] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f2682bb26fc42ac8a2f39e943a79522, State: Initialized, Role: FOLLOWER
I20260810 14:44:31.894493 28230 consensus_queue.cc:260] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:31.895251 28230 ts_tablet_manager.cc:1428] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260810 14:44:31.912818 28150 raft_consensus.cc:482] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:44:31.913057 28150 raft_consensus.cc:504] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:31.913919 28150 leader_election.cc:288] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d013baa9b7d445c8a956138221b53ff (127.24.215.129:45257), c4fca73b7f7b4e3ca3f5348bbba63fcb (127.24.215.131:37949)
I20260810 14:44:31.914585 27696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "933a8b4a05d849ff82318a743304e0b2" candidate_uuid: "1f2682bb26fc42ac8a2f39e943a79522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d013baa9b7d445c8a956138221b53ff" is_pre_election: true
I20260810 14:44:31.914949 27696 raft_consensus.cc:2454] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f2682bb26fc42ac8a2f39e943a79522 in term 0.
I20260810 14:44:31.915299 28018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "933a8b4a05d849ff82318a743304e0b2" candidate_uuid: "1f2682bb26fc42ac8a2f39e943a79522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" is_pre_election: true
I20260810 14:44:31.915575 27769 leader_election.cc:302] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f2682bb26fc42ac8a2f39e943a79522, 3d013baa9b7d445c8a956138221b53ff; no voters:
I20260810 14:44:31.915711 28018 raft_consensus.cc:2454] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1f2682bb26fc42ac8a2f39e943a79522 in term 0.
I20260810 14:44:31.915942 28150 raft_consensus.cc:2781] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:44:31.916146 28150 raft_consensus.cc:482] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:44:31.916265 28150 raft_consensus.cc:3037] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:31.918573 28150 raft_consensus.cc:504] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:31.919134 28150 leader_election.cc:288] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [CANDIDATE]: Term 1 election: Requested vote from peers 3d013baa9b7d445c8a956138221b53ff (127.24.215.129:45257), c4fca73b7f7b4e3ca3f5348bbba63fcb (127.24.215.131:37949)
I20260810 14:44:31.919649 27696 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "933a8b4a05d849ff82318a743304e0b2" candidate_uuid: "1f2682bb26fc42ac8a2f39e943a79522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d013baa9b7d445c8a956138221b53ff"
I20260810 14:44:31.919710 28018 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "933a8b4a05d849ff82318a743304e0b2" candidate_uuid: "1f2682bb26fc42ac8a2f39e943a79522" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb"
I20260810 14:44:31.919927 27696 raft_consensus.cc:3037] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:31.920084 28018 raft_consensus.cc:3037] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:31.922479 28018 raft_consensus.cc:2454] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f2682bb26fc42ac8a2f39e943a79522 in term 1.
I20260810 14:44:31.922744 27696 raft_consensus.cc:2454] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1f2682bb26fc42ac8a2f39e943a79522 in term 1.
I20260810 14:44:31.923024 27772 leader_election.cc:302] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f2682bb26fc42ac8a2f39e943a79522, c4fca73b7f7b4e3ca3f5348bbba63fcb; no voters:
I20260810 14:44:31.923478 28150 raft_consensus.cc:2781] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:31.923666 28150 raft_consensus.cc:686] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 LEADER]: Becoming Leader. State: Replica: 1f2682bb26fc42ac8a2f39e943a79522, State: Running, Role: LEADER
I20260810 14:44:31.923969 28150 consensus_queue.cc:237] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:31.927278 27503 catalog_manager.cc:5495] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1f2682bb26fc42ac8a2f39e943a79522 (127.24.215.130). New cstate: current_term: 1 leader_uuid: "1f2682bb26fc42ac8a2f39e943a79522" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } health_report { overall_health: UNKNOWN } } }
I20260810 14:44:31.960932 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:31.954751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55272 (local address 127.24.215.130:36427)
0810 14:44:31.955621 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:31.955626 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:31.955646 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:44:31.955697 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:31.955700 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:31.955760 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:31.955843 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:31.956241 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:31.957418 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:31.958857 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:31.959487 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:31.959591 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:31.960089 (+ 498us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:31.960231 (+ 142us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:31.960302 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:31.960477 (+ 175us) server_negotiation.cc:299] Negotiation successful
0810 14:44:31.960540 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":729,"thread_start_us":518,"threads_started":1}
I20260810 14:44:31.970414 27696 raft_consensus.cc:1261] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Refusing update from remote peer 1f2682bb26fc42ac8a2f39e943a79522: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:31.971025 28018 raft_consensus.cc:1261] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Refusing update from remote peer 1f2682bb26fc42ac8a2f39e943a79522: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:31.970978 28150 consensus_queue.cc:1035] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:31.971701 28150 consensus_queue.cc:1035] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:31.984362 28190 mvcc.cc:205] Tried to move back new op lower bound from 7316984102785212416 to 7316984102603079680. Current Snapshot: MvccSnapshot[applied={T|T < 7316984102785212416}]
I20260810 14:44:32.026564 27505 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:44:32.026822 27505 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:44:32.029857 27505 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d: Sending DeleteTablet for 3 replicas of tablet 933a8b4a05d849ff82318a743304e0b2
I20260810 14:44:32.030766 27664 tablet_service.cc:1514] Processing DeleteTablet for tablet 933a8b4a05d849ff82318a743304e0b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:32 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:51394
I20260810 14:44:32.030864 27993 tablet_service.cc:1514] Processing DeleteTablet for tablet 933a8b4a05d849ff82318a743304e0b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:32 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:51416
I20260810 14:44:32.031117 28214 tablet_replica.cc:330] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff: stopping tablet replica
I20260810 14:44:32.031201 28215 tablet_replica.cc:330] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb: stopping tablet replica
I20260810 14:44:32.031147 27832 tablet_service.cc:1514] Processing DeleteTablet for tablet 933a8b4a05d849ff82318a743304e0b2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:32 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:56124
I20260810 14:44:32.031416 28214 raft_consensus.cc:2229] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:44:32.031493 28215 raft_consensus.cc:2229] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:44:32.031633 28214 raft_consensus.cc:2258] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:32.031738 28215 raft_consensus.cc:2258] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:32.031948 28213 tablet_replica.cc:330] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522: stopping tablet replica
I20260810 14:44:32.032191 28213 raft_consensus.cc:2229] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:44:32.032559 28213 raft_consensus.cc:2258] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:32.033047 28215 ts_tablet_manager.cc:1905] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:32.033277 28214 ts_tablet_manager.cc:1905] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:32.033214 27505 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56108:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:44:32.033916 27505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:44:32.034008 28213 ts_tablet_manager.cc:1905] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:32.040446 28215 ts_tablet_manager.cc:1918] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:44:32.040678 28215 log.cc:1196] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/933a8b4a05d849ff82318a743304e0b2
I20260810 14:44:32.040802 28214 ts_tablet_manager.cc:1918] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:44:32.041095 28214 log.cc:1196] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/933a8b4a05d849ff82318a743304e0b2
I20260810 14:44:32.041137 28215 ts_tablet_manager.cc:1939] T 933a8b4a05d849ff82318a743304e0b2 P c4fca73b7f7b4e3ca3f5348bbba63fcb: Deleting consensus metadata
I20260810 14:44:32.041728 28214 ts_tablet_manager.cc:1939] T 933a8b4a05d849ff82318a743304e0b2 P 3d013baa9b7d445c8a956138221b53ff: Deleting consensus metadata
I20260810 14:44:32.042500 27486 catalog_manager.cc:4841] TS c4fca73b7f7b4e3ca3f5348bbba63fcb (127.24.215.131:37949): tablet 933a8b4a05d849ff82318a743304e0b2 (table test-table [id=f607cbce35774c81aa9e2d30527d8129]) successfully deleted
I20260810 14:44:32.043286 27993 tablet_service.cc:1467] Processing CreateTablet for tablet 7d8ab4d5aad241a0bff2c97fb261dc6c (DEFAULT_TABLE table=test-table [id=c46ddeab6ed949e6b4efac5b00c11709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:32.043740 27993 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d8ab4d5aad241a0bff2c97fb261dc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:32.043628 27832 tablet_service.cc:1467] Processing CreateTablet for tablet 7d8ab4d5aad241a0bff2c97fb261dc6c (DEFAULT_TABLE table=test-table [id=c46ddeab6ed949e6b4efac5b00c11709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:32.044072 27832 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d8ab4d5aad241a0bff2c97fb261dc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:32.044883 28213 ts_tablet_manager.cc:1918] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:44:32.044934 27481 catalog_manager.cc:4841] TS 3d013baa9b7d445c8a956138221b53ff (127.24.215.129:45257): tablet 933a8b4a05d849ff82318a743304e0b2 (table test-table [id=f607cbce35774c81aa9e2d30527d8129]) successfully deleted
I20260810 14:44:32.045418 27664 tablet_service.cc:1467] Processing CreateTablet for tablet 7d8ab4d5aad241a0bff2c97fb261dc6c (DEFAULT_TABLE table=test-table [id=c46ddeab6ed949e6b4efac5b00c11709]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:32.045913 27664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7d8ab4d5aad241a0bff2c97fb261dc6c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:32.047300 28213 log.cc:1196] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/933a8b4a05d849ff82318a743304e0b2
I20260810 14:44:32.048010 28213 ts_tablet_manager.cc:1939] T 933a8b4a05d849ff82318a743304e0b2 P 1f2682bb26fc42ac8a2f39e943a79522: Deleting consensus metadata
I20260810 14:44:32.049556 27484 catalog_manager.cc:4841] TS 1f2682bb26fc42ac8a2f39e943a79522 (127.24.215.130:36427): tablet 933a8b4a05d849ff82318a743304e0b2 (table test-table [id=f607cbce35774c81aa9e2d30527d8129]) successfully deleted
I20260810 14:44:32.050393 28228 tablet_bootstrap.cc:492] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb: Bootstrap starting.
I20260810 14:44:32.052001 28230 tablet_bootstrap.cc:492] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522: Bootstrap starting.
I20260810 14:44:32.052392 28228 tablet_bootstrap.cc:654] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:32.053009 28229 tablet_bootstrap.cc:492] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff: Bootstrap starting.
I20260810 14:44:32.054425 28230 tablet_bootstrap.cc:654] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:32.055248 28229 tablet_bootstrap.cc:654] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:32.056618 28228 tablet_bootstrap.cc:492] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb: No bootstrap required, opened a new log
I20260810 14:44:32.056983 28228 ts_tablet_manager.cc:1397] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260810 14:44:32.057960 28229 tablet_bootstrap.cc:492] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff: No bootstrap required, opened a new log
I20260810 14:44:32.057895 28228 raft_consensus.cc:348] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:32.058132 28229 ts_tablet_manager.cc:1397] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.003s
I20260810 14:44:32.058193 28230 tablet_bootstrap.cc:492] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522: No bootstrap required, opened a new log
I20260810 14:44:32.058158 28228 raft_consensus.cc:374] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:32.058333 28228 raft_consensus.cc:729] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4fca73b7f7b4e3ca3f5348bbba63fcb, State: Initialized, Role: FOLLOWER
I20260810 14:44:32.058349 28230 ts_tablet_manager.cc:1397] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260810 14:44:32.058673 28228 consensus_queue.cc:260] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:32.059118 28229 raft_consensus.cc:348] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:32.059432 28228 ts_tablet_manager.cc:1428] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:44:32.059554 28229 raft_consensus.cc:374] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:32.059695 28229 raft_consensus.cc:729] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d013baa9b7d445c8a956138221b53ff, State: Initialized, Role: FOLLOWER
I20260810 14:44:32.060030 28229 consensus_queue.cc:260] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:32.060635 28229 ts_tablet_manager.cc:1428] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260810 14:44:32.060596 28230 raft_consensus.cc:348] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:32.061065 28230 raft_consensus.cc:374] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:32.061251 28230 raft_consensus.cc:729] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f2682bb26fc42ac8a2f39e943a79522, State: Initialized, Role: FOLLOWER
I20260810 14:44:32.061604 28230 consensus_queue.cc:260] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:32.062220 28230 ts_tablet_manager.cc:1428] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260810 14:44:32.235922 28234 raft_consensus.cc:482] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:44:32.236397 28234 raft_consensus.cc:504] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:32.238374 28234 leader_election.cc:288] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f2682bb26fc42ac8a2f39e943a79522 (127.24.215.130:36427), c4fca73b7f7b4e3ca3f5348bbba63fcb (127.24.215.131:37949)
I20260810 14:44:32.248617 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:32.239029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:38169 (local address 127.24.215.130:36427)
0810 14:44:32.239280 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:32.239288 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:32.239635 (+ 347us) server_negotiation.cc:407] Connection header received
0810 14:44:32.240063 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:32.240070 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:32.240176 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:32.240245 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:32.241102 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.242532 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.246055 (+ 3523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.247542 (+ 1487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.247740 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:32.247836 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:32.248221 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 14:44:32.248356 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260810 14:44:32.248715 28248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:32.238836 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:34679 (local address 127.24.215.131:37949)
0810 14:44:32.240626 (+ 1790us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:32.240630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:32.240657 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:44:32.240722 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:32.240725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:32.240788 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:32.240888 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:32.241649 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.243350 (+ 1701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.246329 (+ 2979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.247665 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.247840 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:32.247915 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:32.248221 (+ 306us) server_negotiation.cc:299] Negotiation successful
0810 14:44:32.248399 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1678,"thread_start_us":1517,"threads_started":1}
I20260810 14:44:32.248864 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:32.238759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:37949 (local address 127.24.215.129:34679)
0810 14:44:32.240082 (+ 1323us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:32.240098 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:32.240202 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:32.240942 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:32.240946 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:32.240988 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:32.241433 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:32.241442 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.243516 (+ 2074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.243522 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:32.246108 (+ 2586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:32.246120 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.247848 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.247854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:32.248073 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:32.248101 (+ 28us) client_negotiation.cc:769] Sending connection context
0810 14:44:32.248180 (+ 79us) client_negotiation.cc:240] Negotiation successful
0810 14:44:32.248352 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1214,"mutex_wait_us":61,"thread_start_us":1036,"threads_started":1}
I20260810 14:44:32.248888 28246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:32.238543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:36427 (local address 127.24.215.129:38169)
0810 14:44:32.239496 (+ 953us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:32.239518 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:32.239832 (+ 314us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:32.240509 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:32.240512 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:32.240535 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:32.240892 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:32.240905 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.242707 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.242714 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:32.245852 (+ 3138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:32.245868 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.247759 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.247765 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:32.247985 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:32.248062 (+ 77us) client_negotiation.cc:769] Sending connection context
0810 14:44:32.248169 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 14:44:32.248352 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":800,"mutex_wait_us":77,"thread_start_us":767,"threads_started":1}
I20260810 14:44:32.249603 28016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d8ab4d5aad241a0bff2c97fb261dc6c" candidate_uuid: "3d013baa9b7d445c8a956138221b53ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" is_pre_election: true
I20260810 14:44:32.249739 27858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d8ab4d5aad241a0bff2c97fb261dc6c" candidate_uuid: "3d013baa9b7d445c8a956138221b53ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f2682bb26fc42ac8a2f39e943a79522" is_pre_election: true
I20260810 14:44:32.249928 28016 raft_consensus.cc:2454] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d013baa9b7d445c8a956138221b53ff in term 0.
I20260810 14:44:32.250023 27858 raft_consensus.cc:2454] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3d013baa9b7d445c8a956138221b53ff in term 0.
I20260810 14:44:32.250495 27606 leader_election.cc:302] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3d013baa9b7d445c8a956138221b53ff, c4fca73b7f7b4e3ca3f5348bbba63fcb; no voters:
I20260810 14:44:32.250950 28234 raft_consensus.cc:2781] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:44:32.251103 28234 raft_consensus.cc:482] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:44:32.251207 28234 raft_consensus.cc:3037] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:32.253556 28234 raft_consensus.cc:504] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:32.254565 27858 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d8ab4d5aad241a0bff2c97fb261dc6c" candidate_uuid: "3d013baa9b7d445c8a956138221b53ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f2682bb26fc42ac8a2f39e943a79522"
I20260810 14:44:32.254629 28016 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7d8ab4d5aad241a0bff2c97fb261dc6c" candidate_uuid: "3d013baa9b7d445c8a956138221b53ff" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb"
I20260810 14:44:32.254796 27858 raft_consensus.cc:3037] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:32.254860 28016 raft_consensus.cc:3037] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:32.254343 28234 leader_election.cc:288] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [CANDIDATE]: Term 1 election: Requested vote from peers 1f2682bb26fc42ac8a2f39e943a79522 (127.24.215.130:36427), c4fca73b7f7b4e3ca3f5348bbba63fcb (127.24.215.131:37949)
I20260810 14:44:32.256978 27858 raft_consensus.cc:2454] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d013baa9b7d445c8a956138221b53ff in term 1.
I20260810 14:44:32.257329 28016 raft_consensus.cc:2454] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3d013baa9b7d445c8a956138221b53ff in term 1.
I20260810 14:44:32.257373 27605 leader_election.cc:302] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f2682bb26fc42ac8a2f39e943a79522, 3d013baa9b7d445c8a956138221b53ff; no voters:
I20260810 14:44:32.257687 28234 raft_consensus.cc:2781] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:32.258188 28234 raft_consensus.cc:686] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [term 1 LEADER]: Becoming Leader. State: Replica: 3d013baa9b7d445c8a956138221b53ff, State: Running, Role: LEADER
I20260810 14:44:32.258601 28234 consensus_queue.cc:237] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } }
I20260810 14:44:32.262215 27505 catalog_manager.cc:5495] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff reported cstate change: term changed from 0 to 1, leader changed from <none> to 3d013baa9b7d445c8a956138221b53ff (127.24.215.129). New cstate: current_term: 1 leader_uuid: "3d013baa9b7d445c8a956138221b53ff" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3d013baa9b7d445c8a956138221b53ff" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45257 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 } health_report { overall_health: UNKNOWN } } }
I20260810 14:44:32.309664 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:32.303187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51744 (local address 127.24.215.129:45257)
0810 14:44:32.303938 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:32.303942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:32.303961 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:44:32.304044 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:32.304047 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:32.304123 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:32.304300 (+ 177us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:32.304760 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.305607 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.307077 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.307590 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.307729 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:32.308220 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:32.308943 (+ 723us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:32.309112 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:32.309212 (+ 100us) server_negotiation.cc:299] Negotiation successful
0810 14:44:32.309299 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":451,"threads_started":1}
I20260810 14:44:32.326449 28254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:32.320462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:36427 (local address 127.24.215.129:35155)
0810 14:44:32.321110 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:32.321126 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:32.321239 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:32.321565 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:32.321568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:32.321578 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:44:32.321865 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:32.321872 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.323336 (+ 1464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.323339 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:32.324398 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:32.324409 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.325364 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.325370 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:32.325529 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:32.325580 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:44:32.325969 (+ 389us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:44:32.325977 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 14:44:32.326035 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 14:44:32.326160 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"thread_start_us":295,"threads_started":1}
I20260810 14:44:32.326478 28239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:32.320601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:35155 (local address 127.24.215.130:36427)
0810 14:44:32.320781 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:32.320787 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:32.321216 (+ 429us) server_negotiation.cc:407] Connection header received
0810 14:44:32.321357 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:32.321362 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:32.321445 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:32.321548 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:32.322000 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.323204 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.324536 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.325188 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.325316 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:32.325744 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:32.325867 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:32.325961 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:32.326081 (+ 120us) server_negotiation.cc:299] Negotiation successful
0810 14:44:32.326185 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:44:32.336007 27858 raft_consensus.cc:1261] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 FOLLOWER]: Refusing update from remote peer 3d013baa9b7d445c8a956138221b53ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:32.336135 28016 raft_consensus.cc:1261] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Refusing update from remote peer 3d013baa9b7d445c8a956138221b53ff: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:32.336762 28249 consensus_queue.cc:1035] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4fca73b7f7b4e3ca3f5348bbba63fcb" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 37949 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:32.337188 28234 consensus_queue.cc:1035] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f2682bb26fc42ac8a2f39e943a79522" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36427 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:32.363190 28251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:32.357149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:56747 (local address 127.24.215.129:45257)
0810 14:44:32.357337 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:32.357345 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:32.358119 (+ 774us) server_negotiation.cc:407] Connection header received
0810 14:44:32.358250 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:32.358254 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:32.358315 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:32.358386 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:32.358844 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.359878 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.361717 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.362137 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.362233 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:32.362597 (+ 364us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:32.362690 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:32.362754 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:32.362880 (+ 126us) server_negotiation.cc:299] Negotiation successful
0810 14:44:32.362942 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:44:32.363289 28259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:32.357210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:45257 (local address 127.24.215.130:56747)
0810 14:44:32.358021 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:32.358038 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:32.358145 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:32.358392 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:32.358395 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:32.358403 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:44:32.358680 (+ 277us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:32.358687 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.360028 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.360036 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:32.361590 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:32.361603 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:32.362293 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:32.362298 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:32.362424 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:32.362460 (+ 36us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:44:32.362775 (+ 315us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:44:32.362783 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 14:44:32.362841 (+ 58us) client_negotiation.cc:240] Negotiation successful
0810 14:44:32.362942 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":657,"thread_start_us":531,"threads_started":1}
W20260810 14:44:32.364436 28190 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260810 14:44:33.952608 27503 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:44:33.952847 27503 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56108:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:44:33.958285 27503 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0876ec41ef5742ecab95b58e40ceb03d: Sending DeleteTablet for 3 replicas of tablet 7d8ab4d5aad241a0bff2c97fb261dc6c
I20260810 14:44:33.959185 27664 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d8ab4d5aad241a0bff2c97fb261dc6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:33 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:51394
I20260810 14:44:33.959276 27832 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d8ab4d5aad241a0bff2c97fb261dc6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:33 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:56124
I20260810 14:44:33.959332 27993 tablet_service.cc:1514] Processing DeleteTablet for tablet 7d8ab4d5aad241a0bff2c97fb261dc6c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:33 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:51416
I20260810 14:44:33.960372 28288 tablet_replica.cc:330] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522: stopping tablet replica
I20260810 14:44:33.960516 28289 tablet_replica.cc:330] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb: stopping tablet replica
I20260810 14:44:33.960616 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 27585
I20260810 14:44:33.960736 28289 raft_consensus.cc:2229] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:44:33.960734 28288 raft_consensus.cc:2229] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:44:33.961114 28288 raft_consensus.cc:2258] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:33.961153 28289 raft_consensus.cc:2258] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:33.962250 28288 ts_tablet_manager.cc:1905] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:33.962934 28289 ts_tablet_manager.cc:1905] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:33.974426 28286 tablet_replica.cc:330] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff: stopping tablet replica
I20260810 14:44:33.974831 28286 raft_consensus.cc:2229] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:44:33.975678 28286 raft_consensus.cc:2258] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:33.978574 28289 ts_tablet_manager.cc:1918] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:44:33.978713 28289 log.cc:1196] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/7d8ab4d5aad241a0bff2c97fb261dc6c
I20260810 14:44:33.979300 28286 ts_tablet_manager.cc:1905] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:33.979333 28288 ts_tablet_manager.cc:1918] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:44:33.979504 28288 log.cc:1196] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/7d8ab4d5aad241a0bff2c97fb261dc6c
I20260810 14:44:33.979741 28289 ts_tablet_manager.cc:1939] T 7d8ab4d5aad241a0bff2c97fb261dc6c P c4fca73b7f7b4e3ca3f5348bbba63fcb: Deleting consensus metadata
I20260810 14:44:33.980160 28288 ts_tablet_manager.cc:1939] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 1f2682bb26fc42ac8a2f39e943a79522: Deleting consensus metadata
I20260810 14:44:33.981140 27486 catalog_manager.cc:4841] TS c4fca73b7f7b4e3ca3f5348bbba63fcb (127.24.215.131:37949): tablet 7d8ab4d5aad241a0bff2c97fb261dc6c (table test-table [id=c46ddeab6ed949e6b4efac5b00c11709]) successfully deleted
I20260810 14:44:33.981241 27484 catalog_manager.cc:4841] TS 1f2682bb26fc42ac8a2f39e943a79522 (127.24.215.130:36427): tablet 7d8ab4d5aad241a0bff2c97fb261dc6c (table test-table [id=c46ddeab6ed949e6b4efac5b00c11709]) successfully deleted
I20260810 14:44:33.987777 28286 ts_tablet_manager.cc:1918] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260810 14:44:33.988018 28286 log.cc:1196] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/7d8ab4d5aad241a0bff2c97fb261dc6c
I20260810 14:44:33.988684 28286 ts_tablet_manager.cc:1939] T 7d8ab4d5aad241a0bff2c97fb261dc6c P 3d013baa9b7d445c8a956138221b53ff: Deleting consensus metadata
I20260810 14:44:33.989804 27481 catalog_manager.cc:4841] TS 3d013baa9b7d445c8a956138221b53ff (127.24.215.129:45257): tablet 7d8ab4d5aad241a0bff2c97fb261dc6c (table test-table [id=c46ddeab6ed949e6b4efac5b00c11709]) successfully deleted
Aug 10 14:44:33 dist-test-slave-vkjp krb5kdc[27449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373071, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:44:34.007869 28291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:33.963263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51748 (local address 127.24.215.129:45257)
0810 14:44:33.985290 (+ 22027us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:33.985295 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:33.985351 (+ 56us) server_negotiation.cc:407] Connection header received
0810 14:44:33.985406 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:33.985409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:33.985472 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:33.985689 (+ 217us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:33.987603 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:33.989157 (+ 1554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:33.991339 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:33.991924 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:33.992027 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:34.001224 (+ 9197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:34.001239 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:34.001256 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:34.001292 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:34.004739 (+ 3447us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:34.005520 (+ 781us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:34.005528 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:34.005530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:34.005747 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:34.006162 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:34.006167 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:34.006169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:34.006588 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:34.006988 (+ 400us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:34.007277 (+ 289us) server_negotiation.cc:299] Negotiation successful
0810 14:44:34.007511 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21898,"thread_start_us":21771,"threads_started":1}
W20260810 14:44:34.008548 27739 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:51748
W20260810 14:44:34.009244 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 27585: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:44:34.009490 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 27585
I20260810 14:44:34.037617 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 27748
Aug 10 14:44:34 dist-test-slave-vkjp krb5kdc[27449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373071, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:44:34.053489 28293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:34.038370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55282 (local address 127.24.215.130:36427)
0810 14:44:34.038979 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:34.038984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:34.039001 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:44:34.039055 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:34.039059 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:34.039177 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:34.039270 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:34.040663 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:34.041444 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:34.042585 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:34.042987 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:34.043198 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:34.047557 (+ 4359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:34.047571 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:34.047587 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:34.047626 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:34.050489 (+ 2863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:34.051284 (+ 795us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:34.051291 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:34.051293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:34.051415 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:34.051849 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:34.051856 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:34.051859 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:34.052252 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:34.052552 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:34.052912 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 14:44:34.053145 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":276,"threads_started":1}
W20260810 14:44:34.054219 27901 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:55282
W20260810 14:44:34.054909 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 27748: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:44:34.055111 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 27748
I20260810 14:44:34.086604 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 27913
Aug 10 14:44:34 dist-test-slave-vkjp krb5kdc[27449](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373071, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:44:34.102671 28294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:34.087404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49570 (local address 127.24.215.131:37949)
0810 14:44:34.087885 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:34.087889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:34.087905 (+ 16us) server_negotiation.cc:407] Connection header received
0810 14:44:34.087957 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:34.087960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:34.088023 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:34.088113 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:34.089415 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:34.090139 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:34.091573 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:34.092007 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:34.092111 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:34.096614 (+ 4503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:34.096629 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:34.096647 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:34.096682 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:34.099652 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:34.100373 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:34.100381 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:34.100384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:34.100530 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:34.100968 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:34.100974 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:34.100978 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:34.101396 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:34.101714 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:34.102054 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 14:44:34.102302 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":238,"threads_started":1}
W20260810 14:44:34.103492 28071 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49570
W20260810 14:44:34.104120 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 27913: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:44:34.104311 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 27913
I20260810 14:44:34.131603 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 27464
I20260810 14:44:34.132855 27551 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:34.250540 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 27464
2026-08-10T14:44:34Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (8764 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:44:34 dist-test-slave-vkjp krb5kdc[28299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:44:34 dist-test-slave-vkjp krb5kdc[28299](info): set up 2 sockets
Aug 10 14:44:34 dist-test-slave-vkjp krb5kdc[28299](info): commencing operation
krb5kdc: starting...
W20260810 14:44:35.433051 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.000s sys 0.020s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:44:35 dist-test-slave-vkjp krb5kdc[28299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:44:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:44:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:44:35.646270 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:38865
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:33241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:38865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:35.974664 28314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:35.974942 28314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:35.975148 28314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:35.982254 28314 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:35.982359 28314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:35.982414 28314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:35.982503 28314 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:35.982569 28314 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:44:35.992558 28314 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:33241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:38865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:38865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:35.993908 28314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:35.995257 28314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:36.007381 28321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:36.007578 28324 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:36.008038 28322 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:36.008380 28314 server_base.cc:1034] running on GCE node
I20260810 14:44:36.009215 28314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:36.010598 28314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:36.011828 28314 hybrid_clock.cc:648] HybridClock initialized: now 1786373076011788 us; error 54 us; skew 500 ppm
I20260810 14:44:36.012193 28314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:36 dist-test-slave-vkjp krb5kdc[28299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373076, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:36.015252 28314 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:44:36.016671 28314 webserver.cc:460] Webserver started at http://127.24.215.190:46787/ using document root <none> and password file <none>
I20260810 14:44:36.017366 28314 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:36.017514 28314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:36.017829 28314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:36.020825 28314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "bc51c740fddd4f7fa62a3abebd443bb5"
format_stamp: "Formatted at 2026-08-10 14:44:36 on dist-test-slave-vkjp"
server_key: "6b8d17b1140eb4fa9ea21ccfa472d245"
server_key_iv: "8269174824a6e6b050b159b74e644fbf"
server_key_version: "encryptionkey@0"
I20260810 14:44:36.021598 28314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "bc51c740fddd4f7fa62a3abebd443bb5"
format_stamp: "Formatted at 2026-08-10 14:44:36 on dist-test-slave-vkjp"
server_key: "6b8d17b1140eb4fa9ea21ccfa472d245"
server_key_iv: "8269174824a6e6b050b159b74e644fbf"
server_key_version: "encryptionkey@0"
I20260810 14:44:36.026700 28314 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:44:36.030117 28332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:36.031469 28314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:44:36.031651 28314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "bc51c740fddd4f7fa62a3abebd443bb5"
format_stamp: "Formatted at 2026-08-10 14:44:36 on dist-test-slave-vkjp"
server_key: "6b8d17b1140eb4fa9ea21ccfa472d245"
server_key_iv: "8269174824a6e6b050b159b74e644fbf"
server_key_version: "encryptionkey@0"
I20260810 14:44:36.031841 28314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:36.053332 28314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:36.057614 28314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:36.057909 28314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:36.076587 28314 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:38865
I20260810 14:44:36.076622 28393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:38865 every 8 connection(s)
I20260810 14:44:36.077950 28314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:44:36.081666 28394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:36.087110 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 28314
I20260810 14:44:36.087384 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:44:36.088248 25438 external_mini_cluster.cc:1428] Setting key 41a73d9b3e249ed0b48836e58e58f86f
I20260810 14:44:36.094323 28394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5: Bootstrap starting.
I20260810 14:44:36.098312 28394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:36.099596 28394 log.cc:824] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:36.102475 28394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5: No bootstrap required, opened a new log
Aug 10 14:44:36 dist-test-slave-vkjp krb5kdc[28299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:36.113255 28397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:36.090549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36168 (local address 127.24.215.190:38865)
0810 14:44:36.091410 (+ 861us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:36.091426 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:36.091467 (+ 41us) server_negotiation.cc:407] Connection header received
0810 14:44:36.093224 (+ 1757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:36.093248 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:36.093625 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:36.093932 (+ 307us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:36.095948 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:36.097294 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:36.099223 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:36.099958 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:36.100082 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:36.104680 (+ 4598us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:36.104690 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:36.104710 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:36.104759 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:36.108779 (+ 4020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:36.109460 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:36.109474 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:36.109482 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:36.109683 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:36.110041 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:36.110046 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:36.110049 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:36.110710 (+ 661us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:36.111019 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:36.111404 (+ 385us) server_negotiation.cc:299] Negotiation successful
0810 14:44:36.111893 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":631,"thread_start_us":408,"threads_started":1}
I20260810 14:44:36.113585 28394 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc51c740fddd4f7fa62a3abebd443bb5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 38865 } }
I20260810 14:44:36.114051 28394 raft_consensus.cc:374] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:36.114180 28394 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc51c740fddd4f7fa62a3abebd443bb5, State: Initialized, Role: FOLLOWER
I20260810 14:44:36.114813 28394 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc51c740fddd4f7fa62a3abebd443bb5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 38865 } }
I20260810 14:44:36.115130 28394 raft_consensus.cc:388] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:44:36.115301 28394 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:44:36.115495 28394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:36.117933 28394 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc51c740fddd4f7fa62a3abebd443bb5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 38865 } }
I20260810 14:44:36.118422 28394 leader_election.cc:302] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bc51c740fddd4f7fa62a3abebd443bb5; no voters:
I20260810 14:44:36.119047 28394 leader_election.cc:288] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:44:36.119220 28399 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:36.120644 28399 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [term 1 LEADER]: Becoming Leader. State: Replica: bc51c740fddd4f7fa62a3abebd443bb5, State: Running, Role: LEADER
I20260810 14:44:36.121184 28399 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc51c740fddd4f7fa62a3abebd443bb5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 38865 } }
I20260810 14:44:36.121376 28394 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:44:36.124580 28401 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc51c740fddd4f7fa62a3abebd443bb5. Latest consensus state: current_term: 1 leader_uuid: "bc51c740fddd4f7fa62a3abebd443bb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc51c740fddd4f7fa62a3abebd443bb5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 38865 } } }
I20260810 14:44:36.124985 28401 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:36.124660 28400 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc51c740fddd4f7fa62a3abebd443bb5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bc51c740fddd4f7fa62a3abebd443bb5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 38865 } } }
I20260810 14:44:36.125099 28400 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5 [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:36.128185 28409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:44:36.134028 28409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:44:36.142494 28409 catalog_manager.cc:1347] Generated new cluster ID: b12aa61970a949fbb4d465f6fc85c34e
I20260810 14:44:36.142602 28409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:44:36.165220 28409 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:44:36.166587 28409 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:44:36.179826 28409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc51c740fddd4f7fa62a3abebd443bb5: Generated new TSK 0
I20260810 14:44:36.180661 28409 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:44:36.230801 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:38865
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:33241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:36.555325 28422 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:36.555595 28422 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:36.555692 28422 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:36.562987 28422 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:36.563112 28422 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:36.563227 28422 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:44:36.572872 28422 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:33241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:38865
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:36.574105 28422 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:36.575433 28422 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:36.589325 28428 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:36.589345 28430 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:36.589622 28432 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:36.590138 28422 server_base.cc:1034] running on GCE node
I20260810 14:44:36.590644 28422 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:36.591569 28422 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:36.592768 28422 hybrid_clock.cc:648] HybridClock initialized: now 1786373076592726 us; error 41 us; skew 500 ppm
I20260810 14:44:36.593073 28422 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:36 dist-test-slave-vkjp krb5kdc[28299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373076, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:36.596167 28422 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:44:36.597478 28422 webserver.cc:460] Webserver started at http://127.24.215.129:45307/ using document root <none> and password file <none>
I20260810 14:44:36.598131 28422 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:36.598269 28422 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:36.598572 28422 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:36.601655 28422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "51415004bcf14744ae80b4dde6378cb2"
format_stamp: "Formatted at 2026-08-10 14:44:36 on dist-test-slave-vkjp"
server_key: "7076898e3b5b73b499794a5f0a39d7ab"
server_key_iv: "a990038692d41a95f184fc26c0546d92"
server_key_version: "encryptionkey@0"
I20260810 14:44:36.602608 28422 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "51415004bcf14744ae80b4dde6378cb2"
format_stamp: "Formatted at 2026-08-10 14:44:36 on dist-test-slave-vkjp"
server_key: "7076898e3b5b73b499794a5f0a39d7ab"
server_key_iv: "a990038692d41a95f184fc26c0546d92"
server_key_version: "encryptionkey@0"
I20260810 14:44:36.607947 28422 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260810 14:44:36.611522 28439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:36.613161 28422 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 14:44:36.613336 28422 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "51415004bcf14744ae80b4dde6378cb2"
format_stamp: "Formatted at 2026-08-10 14:44:36 on dist-test-slave-vkjp"
server_key: "7076898e3b5b73b499794a5f0a39d7ab"
server_key_iv: "a990038692d41a95f184fc26c0546d92"
server_key_version: "encryptionkey@0"
I20260810 14:44:36.613540 28422 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:36.633876 28422 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:36.638438 28422 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:36.638752 28422 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:36.639887 28422 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:36.641374 28422 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:36.641484 28422 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:36.641640 28422 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:36.641716 28422 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:36.675215 28422 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:43909
I20260810 14:44:36.675267 28574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:43909 every 8 connection(s)
I20260810 14:44:36.676549 28422 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:44:36.680182 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 28422
I20260810 14:44:36.680452 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:44:36.681244 25438 external_mini_cluster.cc:1428] Setting key 5a5ca3a41171599eb35360752013fd81
Aug 10 14:44:36 dist-test-slave-vkjp krb5kdc[28299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373076, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:36.699492 28578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:36.679240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:39131 (local address 127.24.215.190:38865)
0810 14:44:36.679865 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:36.679870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:36.680837 (+ 967us) server_negotiation.cc:407] Connection header received
0810 14:44:36.683010 (+ 2173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:36.683016 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:36.683130 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:36.683216 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:36.684722 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:36.685989 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:36.687572 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:36.688287 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:36.688437 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:36.692390 (+ 3953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:36.692399 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:36.692407 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:36.692453 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:36.696023 (+ 3570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:36.696778 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:36.696787 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:36.696793 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:36.696959 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:36.697387 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:36.697395 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:36.697401 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:36.697819 (+ 418us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:36.697985 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:36.698723 (+ 738us) server_negotiation.cc:299] Negotiation successful
0810 14:44:36.698989 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":313,"threads_started":1}
I20260810 14:44:36.700182 28579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:36.679784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:38865 (local address 127.24.215.129:39131)
0810 14:44:36.680681 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:36.680727 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:36.682715 (+ 1988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:36.683467 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:36.683475 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:36.683988 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:36.684545 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:36.684562 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:36.686146 (+ 1584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:36.686150 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:36.687428 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:36.687441 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:36.688497 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:36.688502 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:36.688692 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:36.689380 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:36.689406 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:36.692165 (+ 2759us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:36.696178 (+ 4013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:36.696185 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:36.696198 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:36.696627 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:36.697083 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:36.697087 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:36.697089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:36.697263 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:36.697950 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:36.697957 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:36.698280 (+ 323us) client_negotiation.cc:769] Sending connection context
0810 14:44:36.698570 (+ 290us) client_negotiation.cc:240] Negotiation successful
0810 14:44:36.699011 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":314,"threads_started":1}
I20260810 14:44:36.701948 28575 heartbeater.cc:347] Connected to a master server at 127.24.215.190:38865
I20260810 14:44:36.702250 28575 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
I20260810 14:44:36.703044 28575 heartbeater.cc:511] Master 127.24.215.190:38865 requested a full tablet report, sending...
I20260810 14:44:36.705211 28352 ts_manager.cc:194] Registered new tserver with Master: 51415004bcf14744ae80b4dde6378cb2 (127.24.215.129:43909)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
I20260810 14:44:36.706966 28352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:39131
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:44:36.731122 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:38865
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:33241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:37.062563 28583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:37.062934 28583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:37.063134 28583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:37.070465 28583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:37.070634 28583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:37.070828 28583 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:44:37.080915 28583 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:33241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:38865
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:37.082188 28583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:37.083545 28583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:37.097481 28590 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:37.097509 28593 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:37.097699 28591 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:37.098812 28583 server_base.cc:1034] running on GCE node
I20260810 14:44:37.099516 28583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:37.100462 28583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:37.101785 28583 hybrid_clock.cc:648] HybridClock initialized: now 1786373077101733 us; error 46 us; skew 500 ppm
I20260810 14:44:37.102207 28583 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:37 dist-test-slave-vkjp krb5kdc[28299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373077, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:37.105360 28583 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:44:37.106793 28583 webserver.cc:460] Webserver started at http://127.24.215.130:36031/ using document root <none> and password file <none>
I20260810 14:44:37.107568 28583 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:37.107720 28583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:37.108049 28583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:37.110934 28583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "a74591ad8a2d44b6babe5eb0b8fcc412"
format_stamp: "Formatted at 2026-08-10 14:44:37 on dist-test-slave-vkjp"
server_key: "e19f858b53e77a46d1b742da3d2a0d9c"
server_key_iv: "d7d7eb87bb835656eff6e8f6dbcad0bd"
server_key_version: "encryptionkey@0"
I20260810 14:44:37.111781 28583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "a74591ad8a2d44b6babe5eb0b8fcc412"
format_stamp: "Formatted at 2026-08-10 14:44:37 on dist-test-slave-vkjp"
server_key: "e19f858b53e77a46d1b742da3d2a0d9c"
server_key_iv: "d7d7eb87bb835656eff6e8f6dbcad0bd"
server_key_version: "encryptionkey@0"
I20260810 14:44:37.116817 28583 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:44:37.120343 28601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:37.121874 28583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260810 14:44:37.122067 28583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "a74591ad8a2d44b6babe5eb0b8fcc412"
format_stamp: "Formatted at 2026-08-10 14:44:37 on dist-test-slave-vkjp"
server_key: "e19f858b53e77a46d1b742da3d2a0d9c"
server_key_iv: "d7d7eb87bb835656eff6e8f6dbcad0bd"
server_key_version: "encryptionkey@0"
I20260810 14:44:37.122308 28583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:37.170583 28583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:37.174718 28583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:37.175028 28583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:37.176052 28583 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:37.177496 28583 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:37.177572 28583 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:37.177757 28583 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:37.177809 28583 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:37.211485 28583 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:41375
I20260810 14:44:37.211530 28739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:41375 every 8 connection(s)
I20260810 14:44:37.212842 28583 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:44:37.221524 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 28583
I20260810 14:44:37.221740 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:44:37.222254 25438 external_mini_cluster.cc:1428] Setting key cbb5afa179cd506cfb9d68f0170027b6
Aug 10 14:44:37 dist-test-slave-vkjp krb5kdc[28299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373077, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:37.234956 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:37.215487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:47571 (local address 127.24.215.190:38865)
0810 14:44:37.215922 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:37.215927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:37.216989 (+ 1062us) server_negotiation.cc:407] Connection header received
0810 14:44:37.218906 (+ 1917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:37.218920 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:37.219012 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:37.219159 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:37.220623 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:37.221410 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:37.223224 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:37.223873 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:37.224016 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:37.227678 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:37.227688 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:37.227696 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:37.227741 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:37.231338 (+ 3597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:37.232343 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:37.232347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:37.232349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:37.232480 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:37.232946 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:37.232954 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:37.232960 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:37.233345 (+ 385us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:37.233510 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:37.234172 (+ 662us) server_negotiation.cc:299] Negotiation successful
0810 14:44:37.234427 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":211,"threads_started":1}
I20260810 14:44:37.235796 28744 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:37.216080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:38865 (local address 127.24.215.130:47571)
0810 14:44:37.216821 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:37.216866 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:37.218636 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:37.219387 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:37.219395 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:37.219909 (+ 514us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:37.220453 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:37.220469 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:37.221575 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:37.221578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:37.223033 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:37.223047 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:37.224139 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:37.224150 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:37.224410 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:37.225104 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:37.225126 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:37.227434 (+ 2308us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:37.231502 (+ 4068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:37.231514 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:37.231535 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:37.232218 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:37.232601 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:37.232604 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:37.232607 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:37.232826 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:37.233475 (+ 649us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:37.233481 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:37.233803 (+ 322us) client_negotiation.cc:769] Sending connection context
0810 14:44:37.234036 (+ 233us) client_negotiation.cc:240] Negotiation successful
0810 14:44:37.234428 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":323,"threads_started":1}
I20260810 14:44:37.237301 28740 heartbeater.cc:347] Connected to a master server at 127.24.215.190:38865
I20260810 14:44:37.237627 28740 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:37.238365 28740 heartbeater.cc:511] Master 127.24.215.190:38865 requested a full tablet report, sending...
I20260810 14:44:37.239840 28352 ts_manager.cc:194] Registered new tserver with Master: a74591ad8a2d44b6babe5eb0b8fcc412 (127.24.215.130:41375)
I20260810 14:44:37.240666 28352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:47571
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:44:37.270673 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:38865
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:33241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:37.606611 28747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:37.606884 28747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:37.606980 28747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:37.614493 28747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:37.614601 28747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:37.614714 28747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:44:37.624881 28747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:33241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:38865
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:37.626217 28747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:37.627643 28747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:37.641726 28752 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:37.641681 28755 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:37.641752 28753 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:37.642596 28747 server_base.cc:1034] running on GCE node
I20260810 14:44:37.643237 28747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:37.644212 28747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:37.645411 28747 hybrid_clock.cc:648] HybridClock initialized: now 1786373077645364 us; error 39 us; skew 500 ppm
I20260810 14:44:37.645737 28747 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:37 dist-test-slave-vkjp krb5kdc[28299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373077, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:37.648876 28747 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:44:37.650444 28747 webserver.cc:460] Webserver started at http://127.24.215.131:40909/ using document root <none> and password file <none>
I20260810 14:44:37.651229 28747 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:37.651386 28747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:37.651712 28747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:37.654664 28747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "e663364266334e7baf896e8bd293f8ba"
format_stamp: "Formatted at 2026-08-10 14:44:37 on dist-test-slave-vkjp"
server_key: "2258046986eea8217a851bc2544e377c"
server_key_iv: "c8cbc1d618cdcf74844220b165bacdd3"
server_key_version: "encryptionkey@0"
I20260810 14:44:37.655500 28747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "e663364266334e7baf896e8bd293f8ba"
format_stamp: "Formatted at 2026-08-10 14:44:37 on dist-test-slave-vkjp"
server_key: "2258046986eea8217a851bc2544e377c"
server_key_iv: "c8cbc1d618cdcf74844220b165bacdd3"
server_key_version: "encryptionkey@0"
I20260810 14:44:37.660796 28747 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 14:44:37.664304 28762 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:37.665578 28747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 14:44:37.665769 28747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "e663364266334e7baf896e8bd293f8ba"
format_stamp: "Formatted at 2026-08-10 14:44:37 on dist-test-slave-vkjp"
server_key: "2258046986eea8217a851bc2544e377c"
server_key_iv: "c8cbc1d618cdcf74844220b165bacdd3"
server_key_version: "encryptionkey@0"
I20260810 14:44:37.665971 28747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:37.682775 28747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:37.686645 28747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:37.686969 28747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:37.687958 28747 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:37.689262 28747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:37.689368 28747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:37.689529 28747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:37.689622 28747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:37.710786 28575 heartbeater.cc:503] Master 127.24.215.190:38865 was elected leader, sending a full tablet report...
I20260810 14:44:37.723627 28747 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:44473
I20260810 14:44:37.723675 28895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:44473 every 8 connection(s)
I20260810 14:44:37.724898 28747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:44:37.730789 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 28747
I20260810 14:44:37.731029 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:44:37.731643 25438 external_mini_cluster.cc:1428] Setting key 08722e43acc4820b50af31e87e641d56
Aug 10 14:44:37 dist-test-slave-vkjp krb5kdc[28299](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373077, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:37.745139 28743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:37.727618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:57153 (local address 127.24.215.190:38865)
0810 14:44:37.727795 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:37.727800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:37.728933 (+ 1133us) server_negotiation.cc:407] Connection header received
0810 14:44:37.730765 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:37.730772 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:37.730877 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:37.731006 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:37.733110 (+ 2104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:37.733887 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:37.735587 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:37.736030 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:37.736168 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:37.739613 (+ 3445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:37.739623 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:37.739627 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:37.739659 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:37.742052 (+ 2393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:37.742770 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:37.742775 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:37.742780 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:37.742905 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:37.743421 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:37.743426 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:37.743428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:37.743713 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:37.743883 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:37.744572 (+ 689us) server_negotiation.cc:299] Negotiation successful
0810 14:44:37.744804 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 14:44:37.746206 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:37.728074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:38865 (local address 127.24.215.131:57153)
0810 14:44:37.728775 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:37.728812 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:37.730493 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:37.731346 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:37.731359 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:37.732217 (+ 858us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:37.732916 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:37.732950 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:37.734066 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:37.734071 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:37.735432 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:37.735444 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:37.736259 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:37.736266 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:37.736496 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:37.737196 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:37.737219 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:37.739404 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:37.742198 (+ 2794us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:37.742207 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:37.742222 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:37.742636 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:37.743026 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:37.743031 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:37.743035 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:37.743289 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:37.743850 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:37.743858 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:37.744203 (+ 345us) client_negotiation.cc:769] Sending connection context
0810 14:44:37.744461 (+ 258us) client_negotiation.cc:240] Negotiation successful
0810 14:44:37.744998 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":278,"threads_started":1}
I20260810 14:44:37.747788 28896 heartbeater.cc:347] Connected to a master server at 127.24.215.190:38865
I20260810 14:44:37.748100 28896 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:37.748828 28896 heartbeater.cc:511] Master 127.24.215.190:38865 requested a full tablet report, sending...
I20260810 14:44:37.750293 28352 ts_manager.cc:194] Registered new tserver with Master: e663364266334e7baf896e8bd293f8ba (127.24.215.131:44473)
I20260810 14:44:37.751212 28352 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:57153
I20260810 14:44:37.759826 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 14:44:37.794109 28743 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:37.791419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36180 (local address 127.24.215.190:38865)
0810 14:44:37.791673 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:37.791683 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:37.792037 (+ 354us) server_negotiation.cc:407] Connection header received
0810 14:44:37.792258 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:37.792262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:37.792338 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:37.792444 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:37.793780 (+ 1336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36180: BlockingRecv error: recv got EOF from 127.0.0.1:36180 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:44:37.797073 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 28422
W20260810 14:44:37.799686 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 28422: Not authorized: Client connection negotiation failed: client connection to 127.24.215.129:43909: server requires authentication, but client does not have Kerberos credentials available
I20260810 14:44:37.799866 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 28422
I20260810 14:44:37.821957 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 28583
W20260810 14:44:37.824942 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 28583: Not authorized: Client connection negotiation failed: client connection to 127.24.215.130:41375: server requires authentication, but client does not have Kerberos credentials available
I20260810 14:44:37.825136 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 28583
I20260810 14:44:37.844499 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 28747
W20260810 14:44:37.847574 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 28747: Not authorized: Client connection negotiation failed: client connection to 127.24.215.131:44473: server requires authentication, but client does not have Kerberos credentials available
I20260810 14:44:37.847705 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 28747
I20260810 14:44:37.865689 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 28314
I20260810 14:44:37.866950 28389 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:37.970899 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 28314
2026-08-10T14:44:37Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3714 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:44:38 dist-test-slave-vkjp krb5kdc[28916](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:44:38 dist-test-slave-vkjp krb5kdc[28916](info): set up 2 sockets
Aug 10 14:44:38 dist-test-slave-vkjp krb5kdc[28916](info): commencing operation
krb5kdc: starting...
W20260810 14:44:39.165350 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.001s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:44:39 dist-test-slave-vkjp krb5kdc[28916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:44:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:44:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:44:39.374805 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:46411
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:39209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:46411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:39.705148 28930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:39.705544 28930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:39.705699 28930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:39.712939 28930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:39.713083 28930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:39.713192 28930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:39.713297 28930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:39.713377 28930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:44:39.723140 28930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:39209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:46411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:46411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:39.724469 28930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:39.725785 28930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:39.737900 28939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:39.738063 28937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:39.738379 28936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:39.738798 28930 server_base.cc:1034] running on GCE node
I20260810 14:44:39.739624 28930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:39.740949 28930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:39.742164 28930 hybrid_clock.cc:648] HybridClock initialized: now 1786373079742130 us; error 27 us; skew 500 ppm
I20260810 14:44:39.742520 28930 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:39 dist-test-slave-vkjp krb5kdc[28916](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373079, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:39.749147 28930 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:44:39.750813 28930 webserver.cc:460] Webserver started at http://127.24.215.190:34871/ using document root <none> and password file <none>
I20260810 14:44:39.751593 28930 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:39.751747 28930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:39.752076 28930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:39.755204 28930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "898c89eb58514504a0837835a9e9047c"
format_stamp: "Formatted at 2026-08-10 14:44:39 on dist-test-slave-vkjp"
server_key: "8b875df54c68c0acc2529621d0269c35"
server_key_iv: "a745b9d8f5a9c009a04528eff0ab8e00"
server_key_version: "encryptionkey@0"
I20260810 14:44:39.756043 28930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "898c89eb58514504a0837835a9e9047c"
format_stamp: "Formatted at 2026-08-10 14:44:39 on dist-test-slave-vkjp"
server_key: "8b875df54c68c0acc2529621d0269c35"
server_key_iv: "a745b9d8f5a9c009a04528eff0ab8e00"
server_key_version: "encryptionkey@0"
I20260810 14:44:39.761288 28930 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:44:39.764736 28949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:39.766038 28930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 14:44:39.766230 28930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "898c89eb58514504a0837835a9e9047c"
format_stamp: "Formatted at 2026-08-10 14:44:39 on dist-test-slave-vkjp"
server_key: "8b875df54c68c0acc2529621d0269c35"
server_key_iv: "a745b9d8f5a9c009a04528eff0ab8e00"
server_key_version: "encryptionkey@0"
I20260810 14:44:39.766433 28930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:39.777417 28930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:39.781250 28930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:39.781534 28930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:39.800868 28930 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:46411
I20260810 14:44:39.800885 29008 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:46411 every 8 connection(s)
I20260810 14:44:39.802536 28930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:44:39.803448 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 28930
I20260810 14:44:39.803735 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:44:39.804382 25438 external_mini_cluster.cc:1428] Setting key a1ad77df6642ea86e878bc0bfa0cb61f
I20260810 14:44:39.806900 29009 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 14:44:39 dist-test-slave-vkjp krb5kdc[28916](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373079, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:39.820145 29009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c: Bootstrap starting.
I20260810 14:44:39.823359 29009 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:39.824709 29009 log.cc:824] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:39.827630 29012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:39.806302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39862 (local address 127.24.215.190:46411)
0810 14:44:39.807225 (+ 923us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:39.807240 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:39.807281 (+ 41us) server_negotiation.cc:407] Connection header received
0810 14:44:39.808886 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:39.808907 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:39.809282 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:39.809616 (+ 334us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:39.811589 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:39.812952 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:39.814755 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:39.815469 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:39.815598 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:39.820193 (+ 4595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:39.820200 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:39.820213 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:39.820290 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:39.823403 (+ 3113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:39.824255 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:39.824261 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:39.824267 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:39.824432 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:39.824876 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:39.824880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:39.824882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:39.825602 (+ 720us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:39.825953 (+ 351us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:39.826191 (+ 238us) server_negotiation.cc:299] Negotiation successful
0810 14:44:39.826550 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":378,"threads_started":1}
I20260810 14:44:39.827855 29009 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c: No bootstrap required, opened a new log
I20260810 14:44:39.831346 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:44:39.835484 29009 raft_consensus.cc:348] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "898c89eb58514504a0837835a9e9047c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46411 } }
I20260810 14:44:39.835848 29009 raft_consensus.cc:374] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:39.835966 29009 raft_consensus.cc:729] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 898c89eb58514504a0837835a9e9047c, State: Initialized, Role: FOLLOWER
I20260810 14:44:39.836524 29009 consensus_queue.cc:260] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "898c89eb58514504a0837835a9e9047c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46411 } }
I20260810 14:44:39.836755 29009 raft_consensus.cc:388] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:44:39.836900 29009 raft_consensus.cc:482] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:44:39.837092 29009 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:39.840031 29009 raft_consensus.cc:504] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "898c89eb58514504a0837835a9e9047c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46411 } }
I20260810 14:44:39.840622 29009 leader_election.cc:302] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 898c89eb58514504a0837835a9e9047c; no voters:
I20260810 14:44:39.841285 29009 leader_election.cc:288] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:44:39.841398 29017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:39.843374 29017 raft_consensus.cc:686] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [term 1 LEADER]: Becoming Leader. State: Replica: 898c89eb58514504a0837835a9e9047c, State: Running, Role: LEADER
I20260810 14:44:39.843712 29009 sys_catalog.cc:564] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:44:39.844010 29017 consensus_queue.cc:237] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "898c89eb58514504a0837835a9e9047c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46411 } }
I20260810 14:44:39.849113 29019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 898c89eb58514504a0837835a9e9047c. Latest consensus state: current_term: 1 leader_uuid: "898c89eb58514504a0837835a9e9047c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "898c89eb58514504a0837835a9e9047c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46411 } } }
I20260810 14:44:39.849620 29019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:39.850322 29018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "898c89eb58514504a0837835a9e9047c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "898c89eb58514504a0837835a9e9047c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46411 } } }
I20260810 14:44:39.850642 29018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:39.854924 29028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:44:39.862911 29028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:44:39.886696 29028 catalog_manager.cc:1347] Generated new cluster ID: c0d974da1a344a64915a5d4707baa9a4
I20260810 14:44:39.886847 29028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:44:39.925243 29012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:39.912916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39872 (local address 127.24.215.190:46411)
0810 14:44:39.913128 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:39.913135 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:39.915752 (+ 2617us) server_negotiation.cc:407] Connection header received
0810 14:44:39.916065 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:39.916072 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:39.916180 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:39.916266 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:39.917850 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:39.919003 (+ 1153us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:39.921036 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:39.921702 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:39.921829 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:39.923379 (+ 1550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:39.923386 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:39.923391 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:39.924307 (+ 916us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:39.924377 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:39.924415 (+ 38us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:39.924507 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:39.924765 (+ 258us) server_negotiation.cc:299] Negotiation successful
0810 14:44:39.924868 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 14:44:39.950163 29028 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:44:39.951643 29028 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:44:39.961810 29028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 898c89eb58514504a0837835a9e9047c: Generated new TSK 0
I20260810 14:44:39.962647 29028 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:44:39.971177 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 28930
I20260810 14:44:39.972381 29004 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:40.066183 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 28930
2026-08-10T14:44:40Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2090 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:44:40 dist-test-slave-vkjp krb5kdc[29059](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:44:40 dist-test-slave-vkjp krb5kdc[29059](info): set up 2 sockets
Aug 10 14:44:40 dist-test-slave-vkjp krb5kdc[29059](info): commencing operation
krb5kdc: starting...
W20260810 14:44:41.235693 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.004s sys 0.016s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:44:41 dist-test-slave-vkjp krb5kdc[29059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:44:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:44:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:44:41.442673 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:45251
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:45251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:41.768462 29073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:41.768812 29073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:41.768975 29073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:41.776223 29073 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:41.776371 29073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:41.776500 29073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:41.776605 29073 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:41.776682 29073 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:44:41.786964 29073 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:45251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:45251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:41.788323 29073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:41.789667 29073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:41.801450 29078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:41.801528 29079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:41.801518 29082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:41.802542 29073 server_base.cc:1034] running on GCE node
I20260810 14:44:41.803349 29073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:41.804721 29073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:41.805929 29073 hybrid_clock.cc:648] HybridClock initialized: now 1786373081805878 us; error 35 us; skew 500 ppm
I20260810 14:44:41.806277 29073 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:41 dist-test-slave-vkjp krb5kdc[29059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373081, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:41.809417 29073 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:44:41.810808 29073 webserver.cc:460] Webserver started at http://127.24.215.190:37279/ using document root <none> and password file <none>
I20260810 14:44:41.811515 29073 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:41.811619 29073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:41.811878 29073 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:41.814682 29073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "705d95a879a6470d9876cb527bc48237"
format_stamp: "Formatted at 2026-08-10 14:44:41 on dist-test-slave-vkjp"
server_key: "bcc6f63b64c4f3d9ca90bbc3bb9568be"
server_key_iv: "40e3a7528e7846a602dc0793069e39e5"
server_key_version: "encryptionkey@0"
I20260810 14:44:41.815470 29073 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "705d95a879a6470d9876cb527bc48237"
format_stamp: "Formatted at 2026-08-10 14:44:41 on dist-test-slave-vkjp"
server_key: "bcc6f63b64c4f3d9ca90bbc3bb9568be"
server_key_iv: "40e3a7528e7846a602dc0793069e39e5"
server_key_version: "encryptionkey@0"
I20260810 14:44:41.820428 29073 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:44:41.823655 29091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:41.824961 29073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:44:41.825150 29073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "705d95a879a6470d9876cb527bc48237"
format_stamp: "Formatted at 2026-08-10 14:44:41 on dist-test-slave-vkjp"
server_key: "bcc6f63b64c4f3d9ca90bbc3bb9568be"
server_key_iv: "40e3a7528e7846a602dc0793069e39e5"
server_key_version: "encryptionkey@0"
I20260810 14:44:41.825346 29073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:41.840536 29073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:41.845003 29073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:41.845304 29073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:41.863818 29073 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:45251
I20260810 14:44:41.863864 29157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:45251 every 8 connection(s)
I20260810 14:44:41.865228 29073 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:44:41.868882 29158 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:41.871261 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 29073
I20260810 14:44:41.871498 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:44:41.872184 25438 external_mini_cluster.cc:1428] Setting key 96ecdc114eeed9f3e0ba91e991bf4294
I20260810 14:44:41.880378 29158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237: Bootstrap starting.
I20260810 14:44:41.883543 29158 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:41.884555 29158 log.cc:824] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237: Log is configured to *not* fsync() on all Append() calls
Aug 10 14:44:41 dist-test-slave-vkjp krb5kdc[29059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373081, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:41.887538 29158 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237: No bootstrap required, opened a new log
I20260810 14:44:41.893676 29162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:41.874220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34710 (local address 127.24.215.190:45251)
0810 14:44:41.875003 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:41.875014 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:41.875052 (+ 38us) server_negotiation.cc:407] Connection header received
0810 14:44:41.876751 (+ 1699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:41.876773 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:41.877148 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:41.877559 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:41.878973 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:41.880309 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:41.881592 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:41.882280 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:41.882503 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:41.886419 (+ 3916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:41.886428 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:41.886442 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:41.886476 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:41.889572 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:41.890290 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:41.890305 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:41.890311 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:41.890487 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:41.890869 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:41.890874 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:41.890879 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:41.891485 (+ 606us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:41.891829 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:41.892247 (+ 418us) server_negotiation.cc:299] Negotiation successful
0810 14:44:41.892683 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":586,"thread_start_us":373,"threads_started":1}
I20260810 14:44:41.895442 29158 raft_consensus.cc:348] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705d95a879a6470d9876cb527bc48237" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45251 } }
I20260810 14:44:41.895807 29158 raft_consensus.cc:374] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:41.895936 29158 raft_consensus.cc:729] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 705d95a879a6470d9876cb527bc48237, State: Initialized, Role: FOLLOWER
I20260810 14:44:41.896574 29158 consensus_queue.cc:260] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705d95a879a6470d9876cb527bc48237" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45251 } }
I20260810 14:44:41.896862 29158 raft_consensus.cc:388] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:44:41.897025 29158 raft_consensus.cc:482] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:44:41.897197 29158 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:41.899930 29158 raft_consensus.cc:504] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705d95a879a6470d9876cb527bc48237" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45251 } }
I20260810 14:44:41.900511 29158 leader_election.cc:302] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 705d95a879a6470d9876cb527bc48237; no voters:
I20260810 14:44:41.901154 29158 leader_election.cc:288] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:44:41.901299 29164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:41.902033 29164 raft_consensus.cc:686] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [term 1 LEADER]: Becoming Leader. State: Replica: 705d95a879a6470d9876cb527bc48237, State: Running, Role: LEADER
I20260810 14:44:41.902647 29164 consensus_queue.cc:237] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705d95a879a6470d9876cb527bc48237" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45251 } }
I20260810 14:44:41.903462 29158 sys_catalog.cc:564] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:44:41.905278 29166 sys_catalog.cc:455] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 705d95a879a6470d9876cb527bc48237. Latest consensus state: current_term: 1 leader_uuid: "705d95a879a6470d9876cb527bc48237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705d95a879a6470d9876cb527bc48237" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45251 } } }
I20260810 14:44:41.905737 29166 sys_catalog.cc:458] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:41.906581 29165 sys_catalog.cc:455] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "705d95a879a6470d9876cb527bc48237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "705d95a879a6470d9876cb527bc48237" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45251 } } }
I20260810 14:44:41.906968 29165 sys_catalog.cc:458] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237 [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:41.907931 29169 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:44:41.913658 29169 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:44:41.923231 29169 catalog_manager.cc:1347] Generated new cluster ID: 7ff6a45115c14516bc81cd555ae99d26
I20260810 14:44:41.923365 29169 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:44:41.942373 29169 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:44:41.943886 29169 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:44:41.950429 29169 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 705d95a879a6470d9876cb527bc48237: Generated new TSK 0
I20260810 14:44:41.951412 29169 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:44:42.014549 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:45251
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:42.347819 29188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:42.348107 29188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:42.348202 29188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:42.355583 29188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:42.355692 29188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:42.355813 29188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:44:42.365384 29188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:45251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:42.366607 29188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:42.367981 29188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:42.380769 29194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:42.380826 29197 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:42.380847 29195 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:42.382203 29188 server_base.cc:1034] running on GCE node
I20260810 14:44:42.382812 29188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:42.383827 29188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:42.385059 29188 hybrid_clock.cc:648] HybridClock initialized: now 1786373082385021 us; error 50 us; skew 500 ppm
I20260810 14:44:42.385380 29188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:42 dist-test-slave-vkjp krb5kdc[29059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373082, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:42.388654 29188 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:44:42.390129 29188 webserver.cc:460] Webserver started at http://127.24.215.129:39375/ using document root <none> and password file <none>
I20260810 14:44:42.390817 29188 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:42.390978 29188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:42.391351 29188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:42.394265 29188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "46936a3a75e8466ea6d0b9f38b48dca5"
format_stamp: "Formatted at 2026-08-10 14:44:42 on dist-test-slave-vkjp"
server_key: "04363a51d6ba38507a028ef90fb28cc9"
server_key_iv: "0b6261420db448884999fc329aaaf590"
server_key_version: "encryptionkey@0"
I20260810 14:44:42.395051 29188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "46936a3a75e8466ea6d0b9f38b48dca5"
format_stamp: "Formatted at 2026-08-10 14:44:42 on dist-test-slave-vkjp"
server_key: "04363a51d6ba38507a028ef90fb28cc9"
server_key_iv: "0b6261420db448884999fc329aaaf590"
server_key_version: "encryptionkey@0"
I20260810 14:44:42.400539 29188 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 14:44:42.404033 29206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:42.405438 29188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:44:42.405619 29188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "46936a3a75e8466ea6d0b9f38b48dca5"
format_stamp: "Formatted at 2026-08-10 14:44:42 on dist-test-slave-vkjp"
server_key: "04363a51d6ba38507a028ef90fb28cc9"
server_key_iv: "0b6261420db448884999fc329aaaf590"
server_key_version: "encryptionkey@0"
I20260810 14:44:42.405820 29188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:42.455163 29188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:42.459224 29188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:42.459538 29188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:42.460521 29188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:42.461927 29188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:42.462041 29188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:42.462200 29188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:42.462275 29188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:42.496345 29188 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:40579
I20260810 14:44:42.496397 29343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:40579 every 8 connection(s)
I20260810 14:44:42.497681 29188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:44:42.505704 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 29188
I20260810 14:44:42.505930 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:44:42.506673 25438 external_mini_cluster.cc:1428] Setting key 2e1c107bfc90127a5028a4d32598a6e3
Aug 10 14:44:42 dist-test-slave-vkjp krb5kdc[29059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373082, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:42.518944 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:42.500414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:47299 (local address 127.24.215.190:45251)
0810 14:44:42.500856 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:42.500861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:42.501913 (+ 1052us) server_negotiation.cc:407] Connection header received
0810 14:44:42.503775 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:42.503779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:42.503850 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:42.503949 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:42.505410 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:42.506238 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:42.508058 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:42.508581 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:42.508722 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:42.513025 (+ 4303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:42.513031 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:42.513034 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:42.513068 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:42.515784 (+ 2716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:42.516545 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:42.516549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:42.516551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:42.516668 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:42.517132 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:42.517136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:42.517138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:42.517432 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:42.517589 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:42.518327 (+ 738us) server_negotiation.cc:299] Negotiation successful
0810 14:44:42.518565 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":219,"threads_started":1}
I20260810 14:44:42.519958 29348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:42.501055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:45251 (local address 127.24.215.129:47299)
0810 14:44:42.501757 (+ 702us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:42.501794 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:42.503487 (+ 1693us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:42.504156 (+ 669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:42.504164 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:42.504681 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:42.505214 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:42.505230 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:42.506386 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:42.506397 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:42.507894 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:42.507906 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:42.508813 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:42.508821 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:42.509084 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:42.509824 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:42.509868 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:42.512791 (+ 2923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:42.515944 (+ 3153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:42.515953 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:42.515971 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:42.516402 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:42.516785 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:42.516790 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:42.516794 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:42.517002 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:42.517567 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:42.517576 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:42.517936 (+ 360us) client_negotiation.cc:769] Sending connection context
0810 14:44:42.518206 (+ 270us) client_negotiation.cc:240] Negotiation successful
0810 14:44:42.518771 (+ 565us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":270,"threads_started":1}
I20260810 14:44:42.521638 29344 heartbeater.cc:347] Connected to a master server at 127.24.215.190:45251
I20260810 14:44:42.521984 29344 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:42.522948 29344 heartbeater.cc:511] Master 127.24.215.190:45251 requested a full tablet report, sending...
I20260810 14:44:42.525725 29112 ts_manager.cc:194] Registered new tserver with Master: 46936a3a75e8466ea6d0b9f38b48dca5 (127.24.215.129:40579)
I20260810 14:44:42.528029 29112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:47299
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:44:42.555502 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:45251
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:42.881760 29351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:42.882093 29351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:42.882272 29351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:42.889698 29351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:42.889851 29351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:42.890046 29351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:44:42.900693 29351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:45251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:42.902168 29351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:42.903666 29351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:42.917567 29358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:42.917531 29361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:42.917609 29359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:42.918241 29351 server_base.cc:1034] running on GCE node
I20260810 14:44:42.918900 29351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:42.919925 29351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:42.921152 29351 hybrid_clock.cc:648] HybridClock initialized: now 1786373082921111 us; error 52 us; skew 500 ppm
I20260810 14:44:42.921468 29351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:42 dist-test-slave-vkjp krb5kdc[29059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373082, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:42.924667 29351 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:44:42.926075 29351 webserver.cc:460] Webserver started at http://127.24.215.130:44365/ using document root <none> and password file <none>
I20260810 14:44:42.926759 29351 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:42.926905 29351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:42.927270 29351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:42.930214 29351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "d0a124e6b3494282bc4544f2f5fab294"
format_stamp: "Formatted at 2026-08-10 14:44:42 on dist-test-slave-vkjp"
server_key: "b357fd87a6de5dcb11e8d09f72264a81"
server_key_iv: "a5d8072809adfaaaf6d1bae60090b42c"
server_key_version: "encryptionkey@0"
I20260810 14:44:42.930989 29351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "d0a124e6b3494282bc4544f2f5fab294"
format_stamp: "Formatted at 2026-08-10 14:44:42 on dist-test-slave-vkjp"
server_key: "b357fd87a6de5dcb11e8d09f72264a81"
server_key_iv: "a5d8072809adfaaaf6d1bae60090b42c"
server_key_version: "encryptionkey@0"
I20260810 14:44:42.936198 29351 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:44:42.939604 29371 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:42.940852 29351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 14:44:42.941046 29351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "d0a124e6b3494282bc4544f2f5fab294"
format_stamp: "Formatted at 2026-08-10 14:44:42 on dist-test-slave-vkjp"
server_key: "b357fd87a6de5dcb11e8d09f72264a81"
server_key_iv: "a5d8072809adfaaaf6d1bae60090b42c"
server_key_version: "encryptionkey@0"
I20260810 14:44:42.941262 29351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:42.957580 29351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:42.961865 29351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:42.962173 29351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:42.963201 29351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:42.964530 29351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:42.964633 29351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:42.964794 29351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:42.964869 29351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:42.998281 29351 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:44793
I20260810 14:44:42.998453 29506 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:44793 every 8 connection(s)
I20260810 14:44:42.999752 29351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:44:43.006400 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 29351
I20260810 14:44:43.006618 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:44:43.007149 25438 external_mini_cluster.cc:1428] Setting key 997dd7ad8cf477e13bc2fab5580c60ab
Aug 10 14:44:43 dist-test-slave-vkjp krb5kdc[29059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373082, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:43.022177 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:43.002500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:40629 (local address 127.24.215.190:45251)
0810 14:44:43.002693 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:43.002697 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:43.004125 (+ 1428us) server_negotiation.cc:407] Connection header received
0810 14:44:43.006078 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:43.006084 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:43.006172 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:43.006286 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:43.008174 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:43.008990 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:43.011593 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:43.012096 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:43.012241 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:43.016058 (+ 3817us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:43.016063 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:43.016066 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:43.016098 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:43.018805 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:43.019663 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:43.019669 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:43.019677 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:43.019816 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:43.020273 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:43.020277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:43.020279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:43.020559 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:43.020717 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:43.021586 (+ 869us) server_negotiation.cc:299] Negotiation successful
0810 14:44:43.021828 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 14:44:43.023272 29509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:43.003017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:45251 (local address 127.24.215.130:40629)
0810 14:44:43.003952 (+ 935us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:43.003991 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:43.005775 (+ 1784us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:43.006577 (+ 802us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:43.006589 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:43.007292 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:43.008004 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:43.008025 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:43.009191 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:43.009196 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:43.011433 (+ 2237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:43.011453 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:43.012356 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:43.012367 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:43.012706 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:43.013578 (+ 872us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:43.013608 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:43.015814 (+ 2206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:43.018969 (+ 3155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:43.018979 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:43.018997 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:43.019523 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:43.019938 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:43.019942 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:43.019946 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:43.020144 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:43.020690 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:43.020699 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:43.021195 (+ 496us) client_negotiation.cc:769] Sending connection context
0810 14:44:43.021485 (+ 290us) client_negotiation.cc:240] Negotiation successful
0810 14:44:43.022061 (+ 576us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":678,"thread_start_us":399,"threads_started":1}
I20260810 14:44:43.024813 29507 heartbeater.cc:347] Connected to a master server at 127.24.215.190:45251
I20260810 14:44:43.025132 29507 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:43.025936 29507 heartbeater.cc:511] Master 127.24.215.190:45251 requested a full tablet report, sending...
I20260810 14:44:43.027483 29112 ts_manager.cc:194] Registered new tserver with Master: d0a124e6b3494282bc4544f2f5fab294 (127.24.215.130:44793)
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
I20260810 14:44:43.028457 29112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:40629
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:44:43.054586 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:45251
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:42077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:43.386492 29512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:43.386754 29512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:43.386931 29512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:43.394112 29512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:43.394208 29512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:43.394359 29512 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:44:43.404088 29512 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:42077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:45251
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:43.405441 29512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:43.406762 29512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:43.419893 29520 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:43.419962 29517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:43.419925 29518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:43.421000 29512 server_base.cc:1034] running on GCE node
I20260810 14:44:43.421612 29512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:43.422511 29512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:43.423735 29512 hybrid_clock.cc:648] HybridClock initialized: now 1786373083423697 us; error 50 us; skew 500 ppm
I20260810 14:44:43.424055 29512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:43 dist-test-slave-vkjp krb5kdc[29059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373083, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:43.427325 29512 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:44:43.428776 29512 webserver.cc:460] Webserver started at http://127.24.215.131:41587/ using document root <none> and password file <none>
I20260810 14:44:43.429504 29512 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:43.429611 29512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:43.429869 29512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:43.432870 29512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "56eccbba7cd745a39ffa7ac83e4396e9"
format_stamp: "Formatted at 2026-08-10 14:44:43 on dist-test-slave-vkjp"
server_key: "7f8f496b83326c3b3d0b8f5878ab32dc"
server_key_iv: "6dfe35a6880db2642cdc5c61f274a06a"
server_key_version: "encryptionkey@0"
I20260810 14:44:43.433962 29512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "56eccbba7cd745a39ffa7ac83e4396e9"
format_stamp: "Formatted at 2026-08-10 14:44:43 on dist-test-slave-vkjp"
server_key: "7f8f496b83326c3b3d0b8f5878ab32dc"
server_key_iv: "6dfe35a6880db2642cdc5c61f274a06a"
server_key_version: "encryptionkey@0"
I20260810 14:44:43.439050 29512 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 14:44:43.442482 29529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:43.443785 29512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:44:43.443974 29512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "56eccbba7cd745a39ffa7ac83e4396e9"
format_stamp: "Formatted at 2026-08-10 14:44:43 on dist-test-slave-vkjp"
server_key: "7f8f496b83326c3b3d0b8f5878ab32dc"
server_key_iv: "6dfe35a6880db2642cdc5c61f274a06a"
server_key_version: "encryptionkey@0"
I20260810 14:44:43.444173 29512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:43.457365 29512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:43.461603 29512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:43.461907 29512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:43.462831 29512 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:43.464296 29512 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:43.464399 29512 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:43.464578 29512 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:43.464658 29512 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:43.498953 29512 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:41555
I20260810 14:44:43.498989 29671 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:41555 every 8 connection(s)
I20260810 14:44:43.500299 29512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:44:43.505136 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 29512
I20260810 14:44:43.505334 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:44:43.505910 25438 external_mini_cluster.cc:1428] Setting key 55a56341a91846111721a572528118f6
Aug 10 14:44:43 dist-test-slave-vkjp krb5kdc[29059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373083, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:43.521567 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:43.502989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:34535 (local address 127.24.215.190:45251)
0810 14:44:43.503216 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:43.503224 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:43.504470 (+ 1246us) server_negotiation.cc:407] Connection header received
0810 14:44:43.506915 (+ 2445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:43.506921 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:43.507001 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:43.507131 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:43.509104 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:43.510063 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:43.511672 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:43.512345 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:43.512508 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:43.515879 (+ 3371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:43.515884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:43.515887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:43.515919 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:43.518211 (+ 2292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:43.518899 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:43.518904 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:43.518909 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:43.519049 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:43.519580 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:43.519584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:43.519586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:43.519972 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:43.520137 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:43.520951 (+ 814us) server_negotiation.cc:299] Negotiation successful
0810 14:44:43.521201 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 14:44:43.522519 29675 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:43.503542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:45251 (local address 127.24.215.131:34535)
0810 14:44:43.504326 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:43.504362 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:43.506649 (+ 2287us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:43.507404 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:43.507422 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:43.508091 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:43.508917 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:43.508943 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:43.510185 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:43.510189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:43.511547 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:43.511560 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:43.512621 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:43.512629 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:43.512854 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:43.513496 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:43.513518 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:43.515665 (+ 2147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:43.518355 (+ 2690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:43.518364 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:43.518378 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:43.518772 (+ 394us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:43.519189 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:43.519195 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:43.519199 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:43.519449 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:43.520110 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:43.520118 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:43.520513 (+ 395us) client_negotiation.cc:769] Sending connection context
0810 14:44:43.520820 (+ 307us) client_negotiation.cc:240] Negotiation successful
0810 14:44:43.521355 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":310,"threads_started":1}
I20260810 14:44:43.523808 29672 heartbeater.cc:347] Connected to a master server at 127.24.215.190:45251
I20260810 14:44:43.524118 29672 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:43.524828 29672 heartbeater.cc:511] Master 127.24.215.190:45251 requested a full tablet report, sending...
I20260810 14:44:43.526156 29112 ts_manager.cc:194] Registered new tserver with Master: 56eccbba7cd745a39ffa7ac83e4396e9 (127.24.215.131:41555)
I20260810 14:44:43.526889 29112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:34535
I20260810 14:44:43.531354 29344 heartbeater.cc:503] Master 127.24.215.190:45251 was elected leader, sending a full tablet report...
I20260810 14:44:43.533762 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:44:43 dist-test-slave-vkjp krb5kdc[29059](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373083, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 10 14:44:43 dist-test-slave-vkjp krb5kdc[29059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373083, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:43.585150 29347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:43.571513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34712 (local address 127.24.215.190:45251)
0810 14:44:43.571689 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:43.571695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:43.572170 (+ 475us) server_negotiation.cc:407] Connection header received
0810 14:44:43.572333 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:43.572337 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:43.572401 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:43.572504 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:43.573854 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:43.574590 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:43.575856 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:43.576257 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:43.576359 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:43.580173 (+ 3814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:43.580178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:43.580182 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:43.580215 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:43.582575 (+ 2360us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:43.583221 (+ 646us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:43.583225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:43.583228 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:43.583352 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:43.583675 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:43.583679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:43.583681 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:43.583998 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:43.584261 (+ 263us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:43.584586 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 14:44:43.584813 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260810 14:44:43.585784 29112 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:34712
I20260810 14:44:43.587647 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 29188
Aug 10 14:44:43 dist-test-slave-vkjp krb5kdc[29059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373083, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:44:43.616730 29684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:43.589010 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38472 (local address 127.24.215.129:40579)
0810 14:44:43.600033 (+ 11023us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:43.600043 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:43.600074 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:44:43.600145 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:43.600155 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:43.600430 (+ 275us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:43.600598 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:43.602240 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:43.603889 (+ 1649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:43.605630 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:43.606557 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:43.606780 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:43.611307 (+ 4527us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:43.611320 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:43.611336 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:43.611370 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:43.614346 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:43.614987 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:43.614993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:43.614996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:43.615181 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:43.615503 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:43.615506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:43.615508 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:43.615830 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:43.616087 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:43.616265 (+ 178us) server_negotiation.cc:299] Negotiation successful
0810 14:44:43.616455 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10818,"thread_start_us":10625,"threads_started":1}
W20260810 14:44:43.617990 29339 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38472
W20260810 14:44:43.619035 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 29188: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:44:43.619340 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 29188
I20260810 14:44:43.639596 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 29351
Aug 10 14:44:43 dist-test-slave-vkjp krb5kdc[29059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373083, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:44:43.656121 29685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:43.640527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54304 (local address 127.24.215.130:44793)
0810 14:44:43.641125 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:43.641132 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:43.641159 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:44:43.641230 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:43.641237 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:43.641445 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:43.641594 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:43.642784 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:43.643861 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:43.645325 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:43.645897 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:43.646050 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:43.650504 (+ 4454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:43.650516 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:43.650530 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:43.650563 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:43.653322 (+ 2759us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:43.653955 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:43.653963 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:43.653966 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:43.654089 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:43.654471 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:43.654477 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:43.654485 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:43.654841 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:43.655288 (+ 447us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:43.655501 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 14:44:43.655768 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":318,"threads_started":1}
W20260810 14:44:43.657441 29502 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:54304
W20260810 14:44:43.658680 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 29351: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:44:43.658821 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 29351
I20260810 14:44:43.678503 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 29512
Aug 10 14:44:43 dist-test-slave-vkjp krb5kdc[29059](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373083, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:44:43.694988 29686 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:43.679509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35818 (local address 127.24.215.131:41555)
0810 14:44:43.680146 (+ 637us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:43.680152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:43.680190 (+ 38us) server_negotiation.cc:407] Connection header received
0810 14:44:43.680253 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:43.680259 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:43.680467 (+ 208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:43.680594 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:43.681934 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:43.683018 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:43.684296 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:43.684838 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:43.684938 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:43.689408 (+ 4470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:43.689420 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:43.689435 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:43.689471 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:43.692491 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:43.693147 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:43.693154 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:43.693156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:43.693276 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:43.693641 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:43.693644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:43.693646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:43.693982 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:43.694232 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:43.694484 (+ 252us) server_negotiation.cc:299] Negotiation successful
0810 14:44:43.694684 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":320,"threads_started":1}
W20260810 14:44:43.696367 29667 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:35818
W20260810 14:44:43.697503 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 29512: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:44:43.697683 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 29512
I20260810 14:44:43.718925 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 29073
I20260810 14:44:43.720230 29153 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:43.823884 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 29073
2026-08-10T14:44:43Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3761 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:44:43 dist-test-slave-vkjp krb5kdc[29691](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:44:43 dist-test-slave-vkjp krb5kdc[29691](info): set up 2 sockets
Aug 10 14:44:43 dist-test-slave-vkjp krb5kdc[29691](info): commencing operation
krb5kdc: starting...
W20260810 14:44:45.000196 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.131s user 0.003s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:44:45 dist-test-slave-vkjp krb5kdc[29691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:44:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:44:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:44:45.207021 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:46535
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:46369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:46535
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:45.536202 29706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:45.536550 29706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:45.536644 29706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:45.543879 29706 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:45.543995 29706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:45.544047 29706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:45.544107 29706 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:45.544153 29706 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:44:45.553947 29706 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:46369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:46535
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:46535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:45.555287 29706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:45.556607 29706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:45.568595 29714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:45.568562 29718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:45.568573 29715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:45.569608 29706 server_base.cc:1034] running on GCE node
I20260810 14:44:45.570336 29706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:45.571798 29706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:45.573037 29706 hybrid_clock.cc:648] HybridClock initialized: now 1786373085573003 us; error 52 us; skew 500 ppm
I20260810 14:44:45.573395 29706 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:45 dist-test-slave-vkjp krb5kdc[29691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373085, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:45.581604 29706 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:44:45.583244 29706 webserver.cc:460] Webserver started at http://127.24.215.190:34957/ using document root <none> and password file <none>
I20260810 14:44:45.583917 29706 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:45.584021 29706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:45.584288 29706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:45.587502 29706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "6cdbdd9af7404dd197a8d8dd67cec91c"
format_stamp: "Formatted at 2026-08-10 14:44:45 on dist-test-slave-vkjp"
server_key: "0ced6c2efa0633e11abe06cd73e486a2"
server_key_iv: "624b406d392f79d869c85624010718ef"
server_key_version: "encryptionkey@0"
I20260810 14:44:45.588528 29706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "6cdbdd9af7404dd197a8d8dd67cec91c"
format_stamp: "Formatted at 2026-08-10 14:44:45 on dist-test-slave-vkjp"
server_key: "0ced6c2efa0633e11abe06cd73e486a2"
server_key_iv: "624b406d392f79d869c85624010718ef"
server_key_version: "encryptionkey@0"
I20260810 14:44:45.595026 29706 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260810 14:44:45.598542 29727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:45.599884 29706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:44:45.600066 29706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "6cdbdd9af7404dd197a8d8dd67cec91c"
format_stamp: "Formatted at 2026-08-10 14:44:45 on dist-test-slave-vkjp"
server_key: "0ced6c2efa0633e11abe06cd73e486a2"
server_key_iv: "624b406d392f79d869c85624010718ef"
server_key_version: "encryptionkey@0"
I20260810 14:44:45.600279 29706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:45.623557 29706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:45.627789 29706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:45.628103 29706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:45.647917 29706 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:46535
I20260810 14:44:45.647940 29787 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:46535 every 8 connection(s)
I20260810 14:44:45.649399 29706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:44:45.653225 29788 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:45.657203 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 29706
I20260810 14:44:45.657483 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:44:45.658325 25438 external_mini_cluster.cc:1428] Setting key 26c74604d02c19cb30942ce759ceac88
I20260810 14:44:45.665325 29788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c: Bootstrap starting.
I20260810 14:44:45.669039 29788 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:45.670182 29788 log.cc:824] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:45.673045 29788 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c: No bootstrap required, opened a new log
Aug 10 14:44:45 dist-test-slave-vkjp krb5kdc[29691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373085, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:45.681195 29788 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdbdd9af7404dd197a8d8dd67cec91c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46535 } }
I20260810 14:44:45.681603 29788 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:45.681726 29788 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6cdbdd9af7404dd197a8d8dd67cec91c, State: Initialized, Role: FOLLOWER
I20260810 14:44:45.682140 29791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:45.660528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60218 (local address 127.24.215.190:46535)
0810 14:44:45.661369 (+ 841us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:45.661381 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:45.661421 (+ 40us) server_negotiation.cc:407] Connection header received
0810 14:44:45.663781 (+ 2360us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:45.663805 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:45.664198 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:45.664519 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:45.666420 (+ 1901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:45.667843 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:45.669610 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:45.670263 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:45.670485 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:45.674492 (+ 4007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:45.674503 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:45.674522 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:45.674562 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:45.677755 (+ 3193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:45.678462 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:45.678479 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:45.678486 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:45.678661 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:45.679078 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:45.679085 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:45.679088 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:45.679673 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:45.679995 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:45.680509 (+ 514us) server_negotiation.cc:299] Negotiation successful
0810 14:44:45.680922 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":412,"threads_started":1}
I20260810 14:44:45.682380 29788 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdbdd9af7404dd197a8d8dd67cec91c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46535 } }
I20260810 14:44:45.682654 29788 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:44:45.682809 29788 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:44:45.683022 29788 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:45.685479 29788 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdbdd9af7404dd197a8d8dd67cec91c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46535 } }
I20260810 14:44:45.685983 29788 leader_election.cc:302] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6cdbdd9af7404dd197a8d8dd67cec91c; no voters:
I20260810 14:44:45.686583 29788 leader_election.cc:288] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:44:45.686702 29793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:45.687440 29793 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [term 1 LEADER]: Becoming Leader. State: Replica: 6cdbdd9af7404dd197a8d8dd67cec91c, State: Running, Role: LEADER
I20260810 14:44:45.688035 29793 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdbdd9af7404dd197a8d8dd67cec91c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46535 } }
I20260810 14:44:45.688758 29788 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:44:45.690887 29795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6cdbdd9af7404dd197a8d8dd67cec91c. Latest consensus state: current_term: 1 leader_uuid: "6cdbdd9af7404dd197a8d8dd67cec91c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdbdd9af7404dd197a8d8dd67cec91c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46535 } } }
I20260810 14:44:45.691303 29795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:45.691505 29794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6cdbdd9af7404dd197a8d8dd67cec91c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6cdbdd9af7404dd197a8d8dd67cec91c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46535 } } }
I20260810 14:44:45.691921 29794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:45.695222 29802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:44:45.700074 29802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:44:45.710350 29802 catalog_manager.cc:1347] Generated new cluster ID: 85129f38b00549e98bf4b8f72c5f1388
I20260810 14:44:45.710494 29802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:44:45.728927 29802 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:44:45.730333 29802 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:44:45.738298 29802 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6cdbdd9af7404dd197a8d8dd67cec91c: Generated new TSK 0
I20260810 14:44:45.739348 29802 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:44:45.804023 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46535
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:46369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:46.143298 29819 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:46.143585 29819 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:46.143754 29819 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:46.151062 29819 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:46.151250 29819 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:46.151446 29819 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:44:46.161336 29819 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:46369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:46.162657 29819 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:46.164007 29819 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:46.177985 29828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:46.177933 29826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:46.177990 29825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:46.185169 29819 server_base.cc:1034] running on GCE node
I20260810 14:44:46.185930 29819 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:46.186929 29819 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:46.188143 29819 hybrid_clock.cc:648] HybridClock initialized: now 1786373086188093 us; error 38 us; skew 500 ppm
I20260810 14:44:46.188511 29819 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:46 dist-test-slave-vkjp krb5kdc[29691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373086, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:46.191959 29819 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:44:46.193608 29819 webserver.cc:460] Webserver started at http://127.24.215.129:33719/ using document root <none> and password file <none>
I20260810 14:44:46.194375 29819 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:46.194530 29819 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:46.194860 29819 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:46.197880 29819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "3e73cf1035734bde9902eb9497de2c73"
format_stamp: "Formatted at 2026-08-10 14:44:46 on dist-test-slave-vkjp"
server_key: "406c4c624ec8c19ad468a094828b5ec4"
server_key_iv: "353c6b91efe8720f9f5f7e86e6eba747"
server_key_version: "encryptionkey@0"
I20260810 14:44:46.198624 29819 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "3e73cf1035734bde9902eb9497de2c73"
format_stamp: "Formatted at 2026-08-10 14:44:46 on dist-test-slave-vkjp"
server_key: "406c4c624ec8c19ad468a094828b5ec4"
server_key_iv: "353c6b91efe8720f9f5f7e86e6eba747"
server_key_version: "encryptionkey@0"
I20260810 14:44:46.203944 29819 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 14:44:46.207362 29835 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:46.208789 29819 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 14:44:46.208976 29819 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "3e73cf1035734bde9902eb9497de2c73"
format_stamp: "Formatted at 2026-08-10 14:44:46 on dist-test-slave-vkjp"
server_key: "406c4c624ec8c19ad468a094828b5ec4"
server_key_iv: "353c6b91efe8720f9f5f7e86e6eba747"
server_key_version: "encryptionkey@0"
I20260810 14:44:46.209172 29819 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:46.222886 29819 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:46.226675 29819 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:46.226987 29819 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:46.228024 29819 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:46.229336 29819 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:46.229409 29819 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:46.229557 29819 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:46.229627 29819 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:46.263222 29819 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:38377
I20260810 14:44:46.263273 29980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:38377 every 8 connection(s)
I20260810 14:44:46.264582 29819 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:44:46.267215 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 29819
I20260810 14:44:46.267467 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:44:46.268543 25438 external_mini_cluster.cc:1428] Setting key 6a46664864e2ebb0fe428abea8a174ee
Aug 10 14:44:46 dist-test-slave-vkjp krb5kdc[29691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373086, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:46.286249 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:46.267206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:41907 (local address 127.24.215.190:46535)
0810 14:44:46.269350 (+ 2144us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:46.269357 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:46.269381 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:44:46.270827 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:46.270833 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:46.270916 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:46.271042 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:46.272546 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:46.273276 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:46.274864 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:46.275362 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:46.275510 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:46.279129 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:46.279139 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:46.279146 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:46.279193 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:46.282800 (+ 3607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:46.283587 (+ 787us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:46.283595 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:46.283601 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:46.283754 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:46.284187 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:46.284195 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:46.284202 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:46.284585 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:46.284756 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:46.285471 (+ 715us) server_negotiation.cc:299] Negotiation successful
0810 14:44:46.285734 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2019,"thread_start_us":1909,"threads_started":1}
I20260810 14:44:46.286895 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:46.267775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:46535 (local address 127.24.215.129:41907)
0810 14:44:46.268586 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:46.268702 (+ 116us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:46.270561 (+ 1859us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:46.271309 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:46.271317 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:46.271826 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:46.272376 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:46.272392 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:46.273440 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:46.273451 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:46.274716 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:46.274729 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:46.275625 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:46.275632 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:46.275867 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:46.276600 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:46.276623 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:46.278852 (+ 2229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:46.282950 (+ 4098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:46.282957 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:46.282971 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:46.283467 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:46.283871 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:46.283874 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:46.283877 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:46.284056 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:46.284734 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:46.284740 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:46.285073 (+ 333us) client_negotiation.cc:769] Sending connection context
0810 14:44:46.285301 (+ 228us) client_negotiation.cc:240] Negotiation successful
0810 14:44:46.285708 (+ 407us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":536,"thread_start_us":334,"threads_started":1}
I20260810 14:44:46.288594 29981 heartbeater.cc:347] Connected to a master server at 127.24.215.190:46535
I20260810 14:44:46.288923 29981 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:46.289769 29981 heartbeater.cc:511] Master 127.24.215.190:46535 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
I20260810 14:44:46.292094 29747 ts_manager.cc:194] Registered new tserver with Master: 3e73cf1035734bde9902eb9497de2c73 (127.24.215.129:38377)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
I20260810 14:44:46.293764 29747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:41907
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:44:46.318925 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46535
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:46369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:46.655468 29987 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:46.655777 29987 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:46.655951 29987 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:46.663313 29987 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:46.663461 29987 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:46.663671 29987 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:44:46.673460 29987 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:46369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:46.674779 29987 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:46.676152 29987 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:46.689612 29994 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:46.689582 29995 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:46.689534 29997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:46.690541 29987 server_base.cc:1034] running on GCE node
I20260810 14:44:46.691218 29987 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:46.692173 29987 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:46.693413 29987 hybrid_clock.cc:648] HybridClock initialized: now 1786373086693363 us; error 33 us; skew 500 ppm
I20260810 14:44:46.693794 29987 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:46 dist-test-slave-vkjp krb5kdc[29691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373086, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:46.697110 29987 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:44:46.698738 29987 webserver.cc:460] Webserver started at http://127.24.215.130:46647/ using document root <none> and password file <none>
I20260810 14:44:46.699522 29987 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:46.699671 29987 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:46.699999 29987 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:46.702929 29987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "a922cbe33a9e4ce5867dc6febc5fed01"
format_stamp: "Formatted at 2026-08-10 14:44:46 on dist-test-slave-vkjp"
server_key: "f3d3c4b40794caef2bc1918392de2c11"
server_key_iv: "04cc84785c384a53ad9dfe2e67a12715"
server_key_version: "encryptionkey@0"
I20260810 14:44:46.703842 29987 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "a922cbe33a9e4ce5867dc6febc5fed01"
format_stamp: "Formatted at 2026-08-10 14:44:46 on dist-test-slave-vkjp"
server_key: "f3d3c4b40794caef2bc1918392de2c11"
server_key_iv: "04cc84785c384a53ad9dfe2e67a12715"
server_key_version: "encryptionkey@0"
I20260810 14:44:46.709323 29987 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:44:46.713011 30005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:46.714489 29987 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 14:44:46.714676 29987 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "a922cbe33a9e4ce5867dc6febc5fed01"
format_stamp: "Formatted at 2026-08-10 14:44:46 on dist-test-slave-vkjp"
server_key: "f3d3c4b40794caef2bc1918392de2c11"
server_key_iv: "04cc84785c384a53ad9dfe2e67a12715"
server_key_version: "encryptionkey@0"
I20260810 14:44:46.714902 29987 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:46.736439 29987 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:46.740638 29987 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:46.740939 29987 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:46.741881 29987 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:46.743248 29987 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:46.743352 29987 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:46.743492 29987 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:46.743568 29987 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:46.777525 29987 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:39793
I20260810 14:44:46.777614 30137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:39793 every 8 connection(s)
I20260810 14:44:46.778851 29987 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:44:46.779834 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 29987
I20260810 14:44:46.780035 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:44:46.780556 25438 external_mini_cluster.cc:1428] Setting key d9f9ee9e2dbee0c501ebbba9b8f4063b
Aug 10 14:44:46 dist-test-slave-vkjp krb5kdc[29691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373086, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:46.801453 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:46.781609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:33381 (local address 127.24.215.190:46535)
0810 14:44:46.781808 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:46.781814 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:46.783249 (+ 1435us) server_negotiation.cc:407] Connection header received
0810 14:44:46.785211 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:46.785217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:46.785312 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:46.785426 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:46.786963 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:46.787893 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:46.790233 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:46.790673 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:46.790776 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:46.795420 (+ 4644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:46.795426 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:46.795429 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:46.795465 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:46.798249 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:46.799020 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:46.799025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:46.799029 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:46.799221 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:46.799671 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:46.799676 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:46.799679 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:46.799960 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:46.800117 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:46.800832 (+ 715us) server_negotiation.cc:299] Negotiation successful
0810 14:44:46.801088 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
I20260810 14:44:46.802527 30140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:46.782198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:46535 (local address 127.24.215.130:33381)
0810 14:44:46.783057 (+ 859us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:46.783141 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:46.784935 (+ 1794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:46.785676 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:46.785683 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:46.786209 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:46.786787 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:46.786804 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:46.788053 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:46.788058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:46.790080 (+ 2022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:46.790100 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:46.790941 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:46.790964 (+ 23us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:46.791388 (+ 424us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:46.792151 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:46.792186 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:46.795157 (+ 2971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:46.798399 (+ 3242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:46.798409 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:46.798428 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:46.798880 (+ 452us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:46.799339 (+ 459us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:46.799344 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:46.799348 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:46.799544 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:46.800090 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:46.800098 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:46.800460 (+ 362us) client_negotiation.cc:769] Sending connection context
0810 14:44:46.800743 (+ 283us) client_negotiation.cc:240] Negotiation successful
0810 14:44:46.801327 (+ 584us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":617,"thread_start_us":359,"threads_started":1}
I20260810 14:44:46.804164 30138 heartbeater.cc:347] Connected to a master server at 127.24.215.190:46535
I20260810 14:44:46.804517 30138 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
I20260810 14:44:46.805310 30138 heartbeater.cc:511] Master 127.24.215.190:46535 requested a full tablet report, sending...
I20260810 14:44:46.806907 29747 ts_manager.cc:194] Registered new tserver with Master: a922cbe33a9e4ce5867dc6febc5fed01 (127.24.215.130:39793)
I20260810 14:44:46.807809 29747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:33381
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:44:46.830368 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46535
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:46369
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:47.160938 30145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:47.161229 30145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:47.161329 30145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:47.168682 30145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:47.168792 30145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:47.168929 30145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:44:47.178649 30145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:46369
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46535
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:47.179997 30145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:47.181315 30145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:47.194833 30151 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:47.194916 30155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:47.194873 30152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:47.195767 30145 server_base.cc:1034] running on GCE node
I20260810 14:44:47.196424 30145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:47.197412 30145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:47.198644 30145 hybrid_clock.cc:648] HybridClock initialized: now 1786373087198593 us; error 46 us; skew 500 ppm
I20260810 14:44:47.198963 30145 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:47 dist-test-slave-vkjp krb5kdc[29691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373087, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:47.202346 30145 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:44:47.204030 30145 webserver.cc:460] Webserver started at http://127.24.215.131:41163/ using document root <none> and password file <none>
I20260810 14:44:47.204892 30145 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:47.205044 30145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:47.205420 30145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:47.208629 30145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "91c34ec5f7d54632a9e1d4f406dd20ef"
format_stamp: "Formatted at 2026-08-10 14:44:47 on dist-test-slave-vkjp"
server_key: "b9fb2f03e8a101a445b4a8be4d01ee99"
server_key_iv: "b3d7c317312cf26c1fa086c441680ef0"
server_key_version: "encryptionkey@0"
I20260810 14:44:47.209548 30145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "91c34ec5f7d54632a9e1d4f406dd20ef"
format_stamp: "Formatted at 2026-08-10 14:44:47 on dist-test-slave-vkjp"
server_key: "b9fb2f03e8a101a445b4a8be4d01ee99"
server_key_iv: "b3d7c317312cf26c1fa086c441680ef0"
server_key_version: "encryptionkey@0"
I20260810 14:44:47.215619 30145 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260810 14:44:47.219496 30169 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:47.220839 30145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:44:47.221009 30145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "91c34ec5f7d54632a9e1d4f406dd20ef"
format_stamp: "Formatted at 2026-08-10 14:44:47 on dist-test-slave-vkjp"
server_key: "b9fb2f03e8a101a445b4a8be4d01ee99"
server_key_iv: "b3d7c317312cf26c1fa086c441680ef0"
server_key_version: "encryptionkey@0"
I20260810 14:44:47.221235 30145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:47.241773 30145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:47.246443 30145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:47.246757 30145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:47.247897 30145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:47.249267 30145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:47.249339 30145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:47.249511 30145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:47.249563 30145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:47.283613 30145 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:38467
I20260810 14:44:47.283701 30305 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:38467 every 8 connection(s)
I20260810 14:44:47.284942 30145 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:44:47.291503 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 30145
I20260810 14:44:47.291716 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:44:47.292258 25438 external_mini_cluster.cc:1428] Setting key 93d10529c28b2b8e6f9e8294672bc4b3
I20260810 14:44:47.296756 29981 heartbeater.cc:503] Master 127.24.215.190:46535 was elected leader, sending a full tablet report...
Aug 10 14:44:47 dist-test-slave-vkjp krb5kdc[29691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373087, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:47.306740 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:47.287586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:52267 (local address 127.24.215.190:46535)
0810 14:44:47.287775 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:47.287779 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:47.289000 (+ 1221us) server_negotiation.cc:407] Connection header received
0810 14:44:47.290852 (+ 1852us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:47.290856 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:47.290925 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:47.291039 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:47.293125 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:47.294281 (+ 1156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:47.296262 (+ 1981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:47.296703 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:47.296855 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:47.300559 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:47.300564 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:47.300568 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:47.300598 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:47.303645 (+ 3047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:47.304372 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:47.304377 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:47.304381 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:47.304524 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:47.304975 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:47.304980 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:47.304984 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:47.305262 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:47.305417 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:47.306134 (+ 717us) server_negotiation.cc:299] Negotiation successful
0810 14:44:47.306348 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:44:47.307931 30308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:47.288068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:46535 (local address 127.24.215.131:52267)
0810 14:44:47.288821 (+ 753us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:47.288884 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:47.290582 (+ 1698us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:47.291358 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:47.291368 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:47.292121 (+ 753us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:47.292923 (+ 802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:47.292944 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:47.294445 (+ 1501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:47.294449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:47.296136 (+ 1687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:47.296148 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:47.296955 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:47.296976 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:47.297192 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:47.297987 (+ 795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:47.298015 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:47.300337 (+ 2322us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:47.303780 (+ 3443us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:47.303789 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:47.303806 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:47.304240 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:47.304651 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:47.304655 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:47.304659 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:47.304850 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:47.305398 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:47.305406 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:47.305733 (+ 327us) client_negotiation.cc:769] Sending connection context
0810 14:44:47.306027 (+ 294us) client_negotiation.cc:240] Negotiation successful
0810 14:44:47.306548 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":284,"threads_started":1}
I20260810 14:44:47.309182 30306 heartbeater.cc:347] Connected to a master server at 127.24.215.190:46535
I20260810 14:44:47.309441 30306 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:47.310097 30306 heartbeater.cc:511] Master 127.24.215.190:46535 requested a full tablet report, sending...
I20260810 14:44:47.311542 29747 ts_manager.cc:194] Registered new tserver with Master: 91c34ec5f7d54632a9e1d4f406dd20ef (127.24.215.131:38467)
I20260810 14:44:47.312353 29747 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:52267
I20260810 14:44:47.320247 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:44:47 dist-test-slave-vkjp krb5kdc[29691](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 14:44:47 dist-test-slave-vkjp krb5kdc[29691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:44:47.373054 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:47.357706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48068 (local address 127.24.215.129:38377)
0810 14:44:47.358392 (+ 686us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:47.358400 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:47.358438 (+ 38us) server_negotiation.cc:407] Connection header received
0810 14:44:47.358501 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:47.358508 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:47.358718 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:47.358891 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:47.360432 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:47.361451 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:47.362677 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:47.363331 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:47.363493 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:47.367343 (+ 3850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:47.367355 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:47.367369 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:47.367408 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:47.370163 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:47.370811 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:47.370819 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:47.370821 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:47.370944 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:47.371378 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:47.371383 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:47.371386 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:47.371749 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:47.372072 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:47.372381 (+ 309us) server_negotiation.cc:299] Negotiation successful
0810 14:44:47.372666 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":323,"threads_started":1}
Aug 10 14:44:47 dist-test-slave-vkjp krb5kdc[29691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:47.391238 29983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:47.377419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60220 (local address 127.24.215.190:46535)
0810 14:44:47.377641 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:47.377649 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:47.377738 (+ 89us) server_negotiation.cc:407] Connection header received
0810 14:44:47.377937 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:47.377942 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:47.378009 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:47.378115 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:47.379422 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:47.380175 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:47.381469 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:47.381915 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:47.382056 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:47.385904 (+ 3848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:47.385909 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:47.385913 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:47.385944 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:47.388429 (+ 2485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:47.389109 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:47.389118 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:47.389123 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:47.389293 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:47.389680 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:47.389685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:47.389688 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:47.390040 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:47.390299 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:47.390624 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 14:44:47.390857 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260810 14:44:47.391863 29747 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60220
I20260810 14:44:47.392859 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 29819
I20260810 14:44:47.405170 30315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:47.393697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48082 (local address 127.24.215.129:38377)
0810 14:44:47.393929 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:47.393939 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:47.394249 (+ 310us) server_negotiation.cc:407] Connection header received
0810 14:44:47.394354 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:47.394377 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:47.394478 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:47.394592 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:47.395799 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:47.396474 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:47.397584 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:47.397976 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:47.398069 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:47.400180 (+ 2111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:47.400184 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:47.400188 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:47.400217 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:47.402801 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:47.403443 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:47.403446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:47.403449 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:47.403591 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:47.403943 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:47.403946 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:47.403948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:47.404194 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:47.404425 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:47.404625 (+ 200us) server_negotiation.cc:299] Negotiation successful
0810 14:44:47.404834 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 14:44:47.405741 29975 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:47.487939 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 29819
I20260810 14:44:47.509092 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 29987
Aug 10 14:44:47 dist-test-slave-vkjp krb5kdc[29691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:44:47.525228 30320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:47.510067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48522 (local address 127.24.215.130:39793)
0810 14:44:47.510758 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:47.510769 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:47.510803 (+ 34us) server_negotiation.cc:407] Connection header received
0810 14:44:47.510910 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:47.510918 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:47.511214 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:47.511350 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:47.512495 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:47.513490 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:47.514660 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:47.515275 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:47.515372 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:47.519616 (+ 4244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:47.519628 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:47.519642 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:47.519681 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:47.522651 (+ 2970us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:47.523358 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:47.523365 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:47.523368 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:47.523502 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:47.523833 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:47.523836 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:47.523838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:47.524204 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:47.524457 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:47.524743 (+ 286us) server_negotiation.cc:299] Negotiation successful
0810 14:44:47.524944 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":360,"threads_started":1}
I20260810 14:44:47.526547 30133 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:47.612709 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 29987
I20260810 14:44:47.633052 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 30145
Aug 10 14:44:47 dist-test-slave-vkjp krb5kdc[29691](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373087, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:44:47.650347 30323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:47.634154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50060 (local address 127.24.215.131:38467)
0810 14:44:47.634812 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:47.634820 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:47.634849 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:44:47.634910 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:47.634916 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:47.635197 (+ 281us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:47.635328 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:47.636592 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:47.637633 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:47.639195 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:47.639763 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:47.639866 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:47.644498 (+ 4632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:47.644510 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:47.644525 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:47.644564 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:47.647662 (+ 3098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:47.648299 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:47.648312 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:47.648317 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:47.648485 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:47.648801 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:47.648806 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:47.648810 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:47.649219 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:47.649501 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:47.649697 (+ 196us) server_negotiation.cc:299] Negotiation successful
0810 14:44:47.649955 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":505,"thread_start_us":352,"threads_started":1}
I20260810 14:44:47.651889 30300 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:47.735131 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 30145
I20260810 14:44:47.754509 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 29706
I20260810 14:44:47.755836 29783 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:47.859508 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 29706
2026-08-10T14:44:47Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4037 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:44:47 dist-test-slave-vkjp krb5kdc[30328](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:44:47 dist-test-slave-vkjp krb5kdc[30328](info): set up 2 sockets
Aug 10 14:44:47 dist-test-slave-vkjp krb5kdc[30328](info): commencing operation
krb5kdc: starting...
W20260810 14:44:49.053360 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.146s user 0.003s sys 0.019s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:44:49 dist-test-slave-vkjp krb5kdc[30328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:44:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:44:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:44:49.273109 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:36637
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:36637
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:49.607255 30344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:49.607533 30344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:49.607715 30344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:49.614902 30344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:49.614995 30344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:49.615119 30344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:49.615211 30344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:49.615280 30344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:44:49.625116 30344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:36637
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:36637
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:49.626443 30344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:49.627806 30344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:49.639658 30350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:49.639690 30349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:49.640461 30352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:49.640671 30344 server_base.cc:1034] running on GCE node
I20260810 14:44:49.641569 30344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:49.643046 30344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:49.644306 30344 hybrid_clock.cc:648] HybridClock initialized: now 1786373089644256 us; error 49 us; skew 500 ppm
I20260810 14:44:49.644702 30344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:49 dist-test-slave-vkjp krb5kdc[30328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373089, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:49.647848 30344 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:44:49.649103 30344 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:49.649236 30344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:49.649576 30344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:49.652573 30344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "37b7ed0145d54177835057ce932fedad"
format_stamp: "Formatted at 2026-08-10 14:44:49 on dist-test-slave-vkjp"
server_key: "c42c18824e93ed56823643dca192a0ef"
server_key_iv: "9f8bcf492575512ad958045c974cfae7"
server_key_version: "encryptionkey@0"
I20260810 14:44:49.653403 30344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "37b7ed0145d54177835057ce932fedad"
format_stamp: "Formatted at 2026-08-10 14:44:49 on dist-test-slave-vkjp"
server_key: "c42c18824e93ed56823643dca192a0ef"
server_key_iv: "9f8bcf492575512ad958045c974cfae7"
server_key_version: "encryptionkey@0"
I20260810 14:44:49.658484 30344 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:44:49.661939 30359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:49.663236 30344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:44:49.663411 30344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "37b7ed0145d54177835057ce932fedad"
format_stamp: "Formatted at 2026-08-10 14:44:49 on dist-test-slave-vkjp"
server_key: "c42c18824e93ed56823643dca192a0ef"
server_key_iv: "9f8bcf492575512ad958045c974cfae7"
server_key_version: "encryptionkey@0"
I20260810 14:44:49.663641 30344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:49.678471 30344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:49.683003 30344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:49.683331 30344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:49.700984 30344 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:36637
I20260810 14:44:49.701009 30427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:36637 every 8 connection(s)
I20260810 14:44:49.701948 30344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:44:49.703862 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 30344
I20260810 14:44:49.704089 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:44:49.704756 25438 external_mini_cluster.cc:1428] Setting key ee0632a864b9c77ca81c69f68bb88ac5
I20260810 14:44:49.705804 30428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:49.718132 30428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad: Bootstrap starting.
Aug 10 14:44:49 dist-test-slave-vkjp krb5kdc[30328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:49.721630 30428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:49.722707 30428 log.cc:824] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:49.725497 30428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad: No bootstrap required, opened a new log
I20260810 14:44:49.726722 30431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:49.706837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33650 (local address 127.24.215.190:36637)
0810 14:44:49.707626 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:49.707663 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:49.707717 (+ 54us) server_negotiation.cc:407] Connection header received
0810 14:44:49.709305 (+ 1588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:49.709329 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:49.709712 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:49.710115 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:49.711887 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:49.713201 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:49.714457 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:49.715242 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:49.715468 (+ 226us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:49.719603 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:49.719612 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:49.719630 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:49.719679 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:49.722699 (+ 3020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:49.723444 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:49.723454 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:49.723459 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:49.723643 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:49.724058 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:49.724062 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:49.724064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:49.724570 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:49.724904 (+ 334us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:49.725319 (+ 415us) server_negotiation.cc:299] Negotiation successful
0810 14:44:49.725758 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":366,"threads_started":1}
I20260810 14:44:49.733485 30428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37b7ed0145d54177835057ce932fedad" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 36637 } }
I20260810 14:44:49.733881 30428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:49.733995 30428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 37b7ed0145d54177835057ce932fedad, State: Initialized, Role: FOLLOWER
I20260810 14:44:49.734524 30428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37b7ed0145d54177835057ce932fedad" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 36637 } }
I20260810 14:44:49.734781 30428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:44:49.734946 30428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:44:49.735136 30428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:49.737442 30428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37b7ed0145d54177835057ce932fedad" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 36637 } }
I20260810 14:44:49.737931 30428 leader_election.cc:302] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 37b7ed0145d54177835057ce932fedad; no voters:
I20260810 14:44:49.738607 30428 leader_election.cc:288] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:44:49.738795 30433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:49.740021 30433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [term 1 LEADER]: Becoming Leader. State: Replica: 37b7ed0145d54177835057ce932fedad, State: Running, Role: LEADER
I20260810 14:44:49.740648 30433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37b7ed0145d54177835057ce932fedad" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 36637 } }
I20260810 14:44:49.741057 30428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:44:49.743713 30435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "37b7ed0145d54177835057ce932fedad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37b7ed0145d54177835057ce932fedad" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 36637 } } }
I20260810 14:44:49.744155 30435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:49.743963 30436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [sys.catalog]: SysCatalogTable state changed. Reason: New leader 37b7ed0145d54177835057ce932fedad. Latest consensus state: current_term: 1 leader_uuid: "37b7ed0145d54177835057ce932fedad" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "37b7ed0145d54177835057ce932fedad" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 36637 } } }
I20260810 14:44:49.744328 30436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:49.745908 30442 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:44:49.751346 30442 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:44:49.760604 30442 catalog_manager.cc:1347] Generated new cluster ID: 4af859c52bb54e239ef902a4f2fbd07f
I20260810 14:44:49.760739 30442 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:44:49.787426 30442 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:44:49.788772 30442 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:44:49.799758 30442 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad: Generated new TSK 0
I20260810 14:44:49.800860 30442 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:44:49.862641 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:36637
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:50.197245 30455 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:50.197542 30455 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:50.197641 30455 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:50.205199 30455 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:50.205305 30455 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:50.205426 30455 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:44:50.215227 30455 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:36637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:50.216583 30455 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:50.217914 30455 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:50.231323 30462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:50.231387 30466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:50.231297 30463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:50.232288 30455 server_base.cc:1034] running on GCE node
I20260810 14:44:50.232900 30455 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:50.233752 30455 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:50.234985 30455 hybrid_clock.cc:648] HybridClock initialized: now 1786373090234941 us; error 53 us; skew 500 ppm
I20260810 14:44:50.235340 30455 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:50 dist-test-slave-vkjp krb5kdc[30328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373090, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:50.238507 30455 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:44:50.239722 30455 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:50.239856 30455 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:50.240172 30455 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:50.243455 30455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "ea2fb8554a944145ac5d2faa2a0414ed"
format_stamp: "Formatted at 2026-08-10 14:44:50 on dist-test-slave-vkjp"
server_key: "3d6f876617170761f86c7acada6d451e"
server_key_iv: "4f0baff8922ddf64ecc9002cb8f6a89f"
server_key_version: "encryptionkey@0"
I20260810 14:44:50.244251 30455 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "ea2fb8554a944145ac5d2faa2a0414ed"
format_stamp: "Formatted at 2026-08-10 14:44:50 on dist-test-slave-vkjp"
server_key: "3d6f876617170761f86c7acada6d451e"
server_key_iv: "4f0baff8922ddf64ecc9002cb8f6a89f"
server_key_version: "encryptionkey@0"
I20260810 14:44:50.249614 30455 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:44:50.253120 30473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:50.254359 30455 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:44:50.254542 30455 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "ea2fb8554a944145ac5d2faa2a0414ed"
format_stamp: "Formatted at 2026-08-10 14:44:50 on dist-test-slave-vkjp"
server_key: "3d6f876617170761f86c7acada6d451e"
server_key_iv: "4f0baff8922ddf64ecc9002cb8f6a89f"
server_key_version: "encryptionkey@0"
I20260810 14:44:50.254791 30455 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:50.283264 30455 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:50.287607 30455 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:50.287860 30455 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:50.288841 30455 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:50.290162 30455 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:50.290253 30455 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:50.290334 30455 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:50.290390 30455 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:50.324294 30455 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:44843
I20260810 14:44:50.324481 30614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:44843 every 8 connection(s)
I20260810 14:44:50.325222 30455 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:44:50.334857 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 30455
I20260810 14:44:50.335237 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:44:50.335989 25438 external_mini_cluster.cc:1428] Setting key 1745ad4c3d3d2d4bd24650e0f0476f34
Aug 10 14:44:50 dist-test-slave-vkjp krb5kdc[30328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373090, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:50.347179 30618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:50.327904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:47781 (local address 127.24.215.190:36637)
0810 14:44:50.328388 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:50.328393 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:50.329324 (+ 931us) server_negotiation.cc:407] Connection header received
0810 14:44:50.331262 (+ 1938us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:50.331268 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:50.331380 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:50.331514 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:50.333096 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:50.333834 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:50.335598 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:50.336034 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:50.336186 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:50.340518 (+ 4332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:50.340524 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:50.340527 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:50.340561 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:50.343399 (+ 2838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:50.344377 (+ 978us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:50.344381 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:50.344384 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:50.344501 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:50.345018 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:50.345021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:50.345023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:50.345379 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:50.345566 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:50.346371 (+ 805us) server_negotiation.cc:299] Negotiation successful
0810 14:44:50.346629 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":371,"thread_start_us":272,"threads_started":1}
I20260810 14:44:50.348012 30619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:50.328369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:36637 (local address 127.24.215.129:47781)
0810 14:44:50.329133 (+ 764us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:50.329172 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:50.330919 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:50.331761 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:50.331770 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:50.332316 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:50.332909 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:50.332926 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:50.334015 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:50.334019 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:50.335442 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:50.335455 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:50.336310 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:50.336317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:50.336552 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:50.337303 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:50.337328 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:50.340281 (+ 2953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:50.343561 (+ 3280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:50.343574 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:50.343595 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:50.344256 (+ 661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:50.344611 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:50.344619 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:50.344625 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:50.344900 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:50.345555 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:50.345562 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:50.345957 (+ 395us) client_negotiation.cc:769] Sending connection context
0810 14:44:50.346194 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 14:44:50.346685 (+ 491us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":316,"threads_started":1}
I20260810 14:44:50.349649 30615 heartbeater.cc:347] Connected to a master server at 127.24.215.190:36637
I20260810 14:44:50.349974 30615 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:50.350653 30615 heartbeater.cc:511] Master 127.24.215.190:36637 requested a full tablet report, sending...
I20260810 14:44:50.352835 30385 ts_manager.cc:194] Registered new tserver with Master: ea2fb8554a944145ac5d2faa2a0414ed (127.24.215.129:44843)
I20260810 14:44:50.354775 30385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:47781
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:44:50.386875 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:36637
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:50.716970 30622 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:50.717237 30622 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:50.717411 30622 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:50.724762 30622 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:50.724864 30622 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:50.725075 30622 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:44:50.734758 30622 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:36637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:50.736147 30622 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:50.737479 30622 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:50.750517 30628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:50.750607 30630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:50.750598 30627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:50.751503 30622 server_base.cc:1034] running on GCE node
I20260810 14:44:50.752133 30622 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:50.753181 30622 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:50.754456 30622 hybrid_clock.cc:648] HybridClock initialized: now 1786373090754399 us; error 47 us; skew 500 ppm
I20260810 14:44:50.754830 30622 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:50 dist-test-slave-vkjp krb5kdc[30328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373090, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:50.763816 30622 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:44:50.765138 30622 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:50.765281 30622 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:50.765627 30622 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:50.768822 30622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "2f2c763f57c641e5bebf76078a61f9a0"
format_stamp: "Formatted at 2026-08-10 14:44:50 on dist-test-slave-vkjp"
server_key: "cb5682a4ad90074a95b4e7699b9f8fb1"
server_key_iv: "2ebc82ed363a59e30ad35cde09e2c6fb"
server_key_version: "encryptionkey@0"
I20260810 14:44:50.769697 30622 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "2f2c763f57c641e5bebf76078a61f9a0"
format_stamp: "Formatted at 2026-08-10 14:44:50 on dist-test-slave-vkjp"
server_key: "cb5682a4ad90074a95b4e7699b9f8fb1"
server_key_iv: "2ebc82ed363a59e30ad35cde09e2c6fb"
server_key_version: "encryptionkey@0"
I20260810 14:44:50.775133 30622 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:44:50.778862 30636 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:50.780393 30622 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:44:50.780596 30622 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "2f2c763f57c641e5bebf76078a61f9a0"
format_stamp: "Formatted at 2026-08-10 14:44:50 on dist-test-slave-vkjp"
server_key: "cb5682a4ad90074a95b4e7699b9f8fb1"
server_key_iv: "2ebc82ed363a59e30ad35cde09e2c6fb"
server_key_version: "encryptionkey@0"
I20260810 14:44:50.780843 30622 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:50.810314 30622 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:50.814582 30622 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:50.814894 30622 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:50.815987 30622 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:50.817385 30622 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:50.817509 30622 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:50.817648 30622 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:50.817739 30622 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:50.851982 30622 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:34781
I20260810 14:44:50.852070 30771 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:34781 every 8 connection(s)
I20260810 14:44:50.852851 30622 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:44:50.858853 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 30622
I20260810 14:44:50.859098 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:44:50.859606 25438 external_mini_cluster.cc:1428] Setting key e17ca88e87ba2d60bf9ecd43b1b5a59b
Aug 10 14:44:50 dist-test-slave-vkjp krb5kdc[30328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373090, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:50.875106 30776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:50.855454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:52721 (local address 127.24.215.190:36637)
0810 14:44:50.855938 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:50.855944 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:50.856982 (+ 1038us) server_negotiation.cc:407] Connection header received
0810 14:44:50.858893 (+ 1911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:50.858899 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:50.858998 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:50.859157 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:50.860915 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:50.861912 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:50.863485 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:50.864235 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:50.864382 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:50.868225 (+ 3843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:50.868231 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:50.868234 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:50.868268 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:50.871738 (+ 3470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:50.872481 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:50.872488 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:50.872495 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:50.872673 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:50.873067 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:50.873074 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:50.873080 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:50.873485 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:50.873650 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:50.874297 (+ 647us) server_negotiation.cc:299] Negotiation successful
0810 14:44:50.874585 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":220,"threads_started":1}
I20260810 14:44:50.875736 30777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:50.856116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:36637 (local address 127.24.215.130:52721)
0810 14:44:50.856809 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:50.856846 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:50.858604 (+ 1758us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:50.859370 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:50.859378 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:50.859998 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:50.860720 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:50.860745 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:50.862059 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:50.862063 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:50.863342 (+ 1279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:50.863354 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:50.864436 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:50.864441 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:50.864618 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:50.865302 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:50.865328 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:50.867997 (+ 2669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:50.871896 (+ 3899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:50.871903 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:50.871916 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:50.872346 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:50.872788 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:50.872791 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:50.872793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:50.872957 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:50.873605 (+ 648us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:50.873611 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:50.873924 (+ 313us) client_negotiation.cc:769] Sending connection context
0810 14:44:50.874160 (+ 236us) client_negotiation.cc:240] Negotiation successful
0810 14:44:50.874545 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"thread_start_us":286,"threads_started":1}
I20260810 14:44:50.877219 30772 heartbeater.cc:347] Connected to a master server at 127.24.215.190:36637
I20260810 14:44:50.877569 30772 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:50.878394 30772 heartbeater.cc:511] Master 127.24.215.190:36637 requested a full tablet report, sending...
I20260810 14:44:50.880157 30385 ts_manager.cc:194] Registered new tserver with Master: 2f2c763f57c641e5bebf76078a61f9a0 (127.24.215.130:34781)
I20260810 14:44:50.881006 30385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:52721
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:44:50.910754 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:36637
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:44395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:44:51.237226 30782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:51.237581 30782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:51.237812 30782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:51.245260 30782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:51.245409 30782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:51.245584 30782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:44:51.255374 30782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:36637
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:51.256700 30782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:51.258095 30782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:51.271575 30788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:51.271482 30792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:51.271524 30790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:51.272449 30782 server_base.cc:1034] running on GCE node
I20260810 14:44:51.273082 30782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:51.274027 30782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:51.275249 30782 hybrid_clock.cc:648] HybridClock initialized: now 1786373091275211 us; error 63 us; skew 500 ppm
I20260810 14:44:51.275578 30782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:44:51 dist-test-slave-vkjp krb5kdc[30328](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373091, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:44:51.278615 30782 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:44:51.279808 30782 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:51.279947 30782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:51.280272 30782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:51.283263 30782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "f4b55ae8f1514665a1c54aabb03d15c4"
format_stamp: "Formatted at 2026-08-10 14:44:51 on dist-test-slave-vkjp"
server_key: "1dae029f9a1c5843fdd9991b809e456f"
server_key_iv: "8d87da1ec2878ba52010e6c56ed95f3e"
server_key_version: "encryptionkey@0"
I20260810 14:44:51.284165 30782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "f4b55ae8f1514665a1c54aabb03d15c4"
format_stamp: "Formatted at 2026-08-10 14:44:51 on dist-test-slave-vkjp"
server_key: "1dae029f9a1c5843fdd9991b809e456f"
server_key_iv: "8d87da1ec2878ba52010e6c56ed95f3e"
server_key_version: "encryptionkey@0"
I20260810 14:44:51.289312 30782 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:44:51.292847 30801 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:51.294149 30782 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 14:44:51.294329 30782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "f4b55ae8f1514665a1c54aabb03d15c4"
format_stamp: "Formatted at 2026-08-10 14:44:51 on dist-test-slave-vkjp"
server_key: "1dae029f9a1c5843fdd9991b809e456f"
server_key_iv: "8d87da1ec2878ba52010e6c56ed95f3e"
server_key_version: "encryptionkey@0"
I20260810 14:44:51.294557 30782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:51.312849 30782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:51.316813 30782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:51.317051 30782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:51.317956 30782 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:51.319326 30782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:51.319422 30782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:51.319497 30782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:51.319552 30782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:51.352619 30782 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:42689
I20260810 14:44:51.352669 30939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:42689 every 8 connection(s)
I20260810 14:44:51.353485 30782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:44:51.358729 30615 heartbeater.cc:503] Master 127.24.215.190:36637 was elected leader, sending a full tablet report...
I20260810 14:44:51.361370 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 30782
I20260810 14:44:51.361572 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:44:51.362103 25438 external_mini_cluster.cc:1428] Setting key 378428b5b0367269d7f3b331aab46f45
Aug 10 14:44:51 dist-test-slave-vkjp krb5kdc[30328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373091, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:44:51.374603 30776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:51.356019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:58227 (local address 127.24.215.190:36637)
0810 14:44:51.356210 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:51.356215 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:51.357317 (+ 1102us) server_negotiation.cc:407] Connection header received
0810 14:44:51.359938 (+ 2621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:51.359943 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:51.360060 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:51.360184 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:51.362258 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.363417 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.365480 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.365880 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.365986 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:51.369556 (+ 3570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:51.369561 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:51.369565 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:51.369614 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:51.372084 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:51.372757 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:51.372761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:51.372763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:51.372879 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:51.373242 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:51.373245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:51.373246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:51.373497 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:51.373605 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:51.374224 (+ 619us) server_negotiation.cc:299] Negotiation successful
0810 14:44:51.374383 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:44:51.375739 30942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:51.356472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:36637 (local address 127.24.215.131:58227)
0810 14:44:51.357156 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:51.357192 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:51.359549 (+ 2357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:51.360591 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:51.360602 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:51.361431 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:51.362025 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:51.362048 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.363604 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.363610 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:51.365314 (+ 1704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:51.365334 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.366072 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.366076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:51.366323 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:51.367070 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:44:51.367141 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:51.369352 (+ 2211us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:51.372217 (+ 2865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:51.372224 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:51.372236 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:51.372658 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:51.372977 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:44:51.372980 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:44:51.372982 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:44:51.373149 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:44:51.373603 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:51.373609 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:51.373914 (+ 305us) client_negotiation.cc:769] Sending connection context
0810 14:44:51.374141 (+ 227us) client_negotiation.cc:240] Negotiation successful
0810 14:44:51.374515 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":280,"threads_started":1}
I20260810 14:44:51.377298 30940 heartbeater.cc:347] Connected to a master server at 127.24.215.190:36637
I20260810 14:44:51.377652 30940 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:51.378319 30940 heartbeater.cc:511] Master 127.24.215.190:36637 requested a full tablet report, sending...
I20260810 14:44:51.379776 30385 ts_manager.cc:194] Registered new tserver with Master: f4b55ae8f1514665a1c54aabb03d15c4 (127.24.215.131:42689)
I20260810 14:44:51.380529 30385 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:58227
I20260810 14:44:51.389762 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:44:51.406807 30776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:51.394187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42286 (local address 127.24.215.190:36637)
0810 14:44:51.394399 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:51.394407 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:51.394901 (+ 494us) server_negotiation.cc:407] Connection header received
0810 14:44:51.395034 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:51.395037 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:51.395162 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:51.395243 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:51.396648 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.397348 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.398735 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.399261 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.399397 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:51.401564 (+ 2167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:51.401569 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:51.401572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:51.401600 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:51.404172 (+ 2572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:51.404798 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:51.404806 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:51.404809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:51.404960 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:51.405340 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:51.405348 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:51.405352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:51.405669 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:51.405945 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:51.406257 (+ 312us) server_negotiation.cc:299] Negotiation successful
0810 14:44:51.406473 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 14:44:51.410701 30385 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42286:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:44:51.417426 30385 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:44:51.440882 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:51.429719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60684 (local address 127.24.215.131:42689)
0810 14:44:51.431603 (+ 1884us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:51.431609 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:51.432249 (+ 640us) server_negotiation.cc:407] Connection header received
0810 14:44:51.433178 (+ 929us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:51.433188 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:51.433422 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:51.433619 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:51.434663 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.435853 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.438608 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.439720 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.439916 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:51.440104 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:51.440358 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 14:44:51.440521 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1750,"thread_start_us":1606,"threads_started":1}
I20260810 14:44:51.440912 30962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:51.429598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:42689 (local address 127.0.0.1:60684)
0810 14:44:51.432105 (+ 2507us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:51.432136 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:51.432350 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:51.433782 (+ 1432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:51.433791 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:51.433819 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:51.434398 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:51.434426 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.435996 (+ 1570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.436003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:51.438405 (+ 2402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:51.438421 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.439911 (+ 1490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.439917 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:51.440114 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:51.440165 (+ 51us) client_negotiation.cc:769] Sending connection context
0810 14:44:51.440302 (+ 137us) client_negotiation.cc:240] Negotiation successful
0810 14:44:51.440496 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2321,"mutex_wait_us":56,"thread_start_us":2824,"threads_started":1}
I20260810 14:44:51.442533 30960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:51.430445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:34781 (local address 127.0.0.1:45020)
0810 14:44:51.433076 (+ 2631us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:51.433095 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:51.433211 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:51.433907 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:51.433928 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:51.433952 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:51.434583 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:51.434598 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.436588 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.436593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:51.439427 (+ 2834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:51.439442 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.441611 (+ 2169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.441616 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:51.441801 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:51.441843 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 14:44:51.441938 (+ 95us) client_negotiation.cc:240] Negotiation successful
0810 14:44:51.442107 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2482,"mutex_wait_us":199,"thread_start_us":1361,"threads_started":1}
I20260810 14:44:51.442457 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:51.430684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45020 (local address 127.24.215.130:34781)
0810 14:44:51.431460 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:51.431471 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:51.433330 (+ 1859us) server_negotiation.cc:407] Connection header received
0810 14:44:51.433426 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:51.433434 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:51.433629 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:51.433797 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:51.434818 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.436411 (+ 1593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.439594 (+ 3183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.441153 (+ 1559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.441310 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:51.441482 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:51.441986 (+ 504us) server_negotiation.cc:299] Negotiation successful
0810 14:44:51.442161 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":452,"threads_started":1}
I20260810 14:44:51.443941 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:51.430185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35926 (local address 127.24.215.129:44843)
0810 14:44:51.431159 (+ 974us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:51.431169 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:51.432840 (+ 1671us) server_negotiation.cc:407] Connection header received
0810 14:44:51.433161 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:51.433170 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:51.433426 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:51.433616 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:51.434741 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.436434 (+ 1693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.439617 (+ 3183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.441112 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.441313 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:51.441508 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:51.443471 (+ 1963us) server_negotiation.cc:299] Negotiation successful
0810 14:44:51.443635 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":745,"thread_start_us":566,"threads_started":1}
I20260810 14:44:51.445191 30861 tablet_service.cc:1467] Processing CreateTablet for tablet 462f3f141aea44b48c27dcdaf82b75e1 (DEFAULT_TABLE table=test-table [id=92d0b033167840268538b6edea65da8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:51.445976 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:51.429637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:44843 (local address 127.0.0.1:35926)
0810 14:44:51.432726 (+ 3089us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:51.432743 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:51.432959 (+ 216us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:51.433871 (+ 912us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:51.433881 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:51.433905 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:51.434524 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:51.434564 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.436587 (+ 2023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.436593 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:51.439428 (+ 2835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:51.439442 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.443160 (+ 3718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.443168 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:51.443290 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:51.443312 (+ 22us) client_negotiation.cc:769] Sending connection context
0810 14:44:51.443419 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 14:44:51.443542 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2962,"mutex_wait_us":182,"thread_start_us":3123,"threads_started":1}
I20260810 14:44:51.445959 30691 tablet_service.cc:1467] Processing CreateTablet for tablet 462f3f141aea44b48c27dcdaf82b75e1 (DEFAULT_TABLE table=test-table [id=92d0b033167840268538b6edea65da8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:51.446570 30861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 462f3f141aea44b48c27dcdaf82b75e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:51.449038 30530 tablet_service.cc:1467] Processing CreateTablet for tablet 462f3f141aea44b48c27dcdaf82b75e1 (DEFAULT_TABLE table=test-table [id=92d0b033167840268538b6edea65da8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:51.450079 30691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 462f3f141aea44b48c27dcdaf82b75e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:51.450187 30530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 462f3f141aea44b48c27dcdaf82b75e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:51.457904 30964 tablet_bootstrap.cc:492] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4: Bootstrap starting.
I20260810 14:44:51.461995 30964 tablet_bootstrap.cc:654] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:51.463016 30964 log.cc:824] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:51.464970 30966 tablet_bootstrap.cc:492] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed: Bootstrap starting.
I20260810 14:44:51.466218 30964 tablet_bootstrap.cc:492] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4: No bootstrap required, opened a new log
I20260810 14:44:51.466512 30964 ts_tablet_manager.cc:1397] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 14:44:51.468946 30966 tablet_bootstrap.cc:654] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:51.470273 30966 log.cc:824] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:51.473532 30966 tablet_bootstrap.cc:492] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed: No bootstrap required, opened a new log
I20260810 14:44:51.473809 30966 ts_tablet_manager.cc:1397] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 14:44:51.475855 30968 tablet_bootstrap.cc:492] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0: Bootstrap starting.
I20260810 14:44:51.477597 30964 raft_consensus.cc:348] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b55ae8f1514665a1c54aabb03d15c4" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 42689 } } peers { permanent_uuid: "ea2fb8554a944145ac5d2faa2a0414ed" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44843 } } peers { permanent_uuid: "2f2c763f57c641e5bebf76078a61f9a0" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 34781 } }
I20260810 14:44:51.478058 30964 raft_consensus.cc:374] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:51.478229 30964 raft_consensus.cc:729] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4b55ae8f1514665a1c54aabb03d15c4, State: Initialized, Role: FOLLOWER
I20260810 14:44:51.478919 30968 tablet_bootstrap.cc:654] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:51.478951 30964 consensus_queue.cc:260] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b55ae8f1514665a1c54aabb03d15c4" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 42689 } } peers { permanent_uuid: "ea2fb8554a944145ac5d2faa2a0414ed" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44843 } } peers { permanent_uuid: "2f2c763f57c641e5bebf76078a61f9a0" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 34781 } }
I20260810 14:44:51.480105 30940 heartbeater.cc:503] Master 127.24.215.190:36637 was elected leader, sending a full tablet report...
I20260810 14:44:51.480990 30968 log.cc:824] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:51.482137 30964 ts_tablet_manager.cc:1428] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4: Time spent starting tablet: real 0.015s user 0.011s sys 0.003s
I20260810 14:44:51.484069 30966 raft_consensus.cc:348] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b55ae8f1514665a1c54aabb03d15c4" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 42689 } } peers { permanent_uuid: "ea2fb8554a944145ac5d2faa2a0414ed" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44843 } } peers { permanent_uuid: "2f2c763f57c641e5bebf76078a61f9a0" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 34781 } }
I20260810 14:44:51.484567 30966 raft_consensus.cc:374] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:51.484678 30966 raft_consensus.cc:729] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea2fb8554a944145ac5d2faa2a0414ed, State: Initialized, Role: FOLLOWER
I20260810 14:44:51.485313 30968 tablet_bootstrap.cc:492] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0: No bootstrap required, opened a new log
I20260810 14:44:51.485276 30966 consensus_queue.cc:260] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b55ae8f1514665a1c54aabb03d15c4" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 42689 } } peers { permanent_uuid: "ea2fb8554a944145ac5d2faa2a0414ed" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44843 } } peers { permanent_uuid: "2f2c763f57c641e5bebf76078a61f9a0" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 34781 } }
I20260810 14:44:51.485715 30968 ts_tablet_manager.cc:1397] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0: Time spent bootstrapping tablet: real 0.010s user 0.009s sys 0.000s
I20260810 14:44:51.487051 30966 ts_tablet_manager.cc:1428] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260810 14:44:51.493758 30968 raft_consensus.cc:348] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b55ae8f1514665a1c54aabb03d15c4" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 42689 } } peers { permanent_uuid: "ea2fb8554a944145ac5d2faa2a0414ed" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44843 } } peers { permanent_uuid: "2f2c763f57c641e5bebf76078a61f9a0" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 34781 } }
I20260810 14:44:51.494138 30968 raft_consensus.cc:374] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:51.494256 30968 raft_consensus.cc:729] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f2c763f57c641e5bebf76078a61f9a0, State: Initialized, Role: FOLLOWER
I20260810 14:44:51.494859 30968 consensus_queue.cc:260] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b55ae8f1514665a1c54aabb03d15c4" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 42689 } } peers { permanent_uuid: "ea2fb8554a944145ac5d2faa2a0414ed" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44843 } } peers { permanent_uuid: "2f2c763f57c641e5bebf76078a61f9a0" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 34781 } }
I20260810 14:44:51.495888 30772 heartbeater.cc:503] Master 127.24.215.190:36637 was elected leader, sending a full tablet report...
I20260810 14:44:51.496340 30968 ts_tablet_manager.cc:1428] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
W20260810 14:44:51.577728 30617 tablet.cc:2369] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:44:51.604791 30773 tablet.cc:2369] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:44:51.605146 30941 tablet.cc:2369] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:44:51.632012 30972 raft_consensus.cc:482] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:44:51.632352 30972 raft_consensus.cc:504] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b55ae8f1514665a1c54aabb03d15c4" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 42689 } } peers { permanent_uuid: "ea2fb8554a944145ac5d2faa2a0414ed" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44843 } } peers { permanent_uuid: "2f2c763f57c641e5bebf76078a61f9a0" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 34781 } }
I20260810 14:44:51.634521 30972 leader_election.cc:288] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4b55ae8f1514665a1c54aabb03d15c4 (127.24.215.131:42689), 2f2c763f57c641e5bebf76078a61f9a0 (127.24.215.130:34781)
I20260810 14:44:51.642699 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:51.635571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:55709 (local address 127.24.215.130:34781)
0810 14:44:51.635794 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:51.635800 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:51.635822 (+ 22us) server_negotiation.cc:407] Connection header received
0810 14:44:51.635903 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:51.635908 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:51.635999 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:51.636112 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:51.636648 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.637364 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.640114 (+ 2750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.641420 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.641595 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:51.641671 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:51.642292 (+ 621us) server_negotiation.cc:299] Negotiation successful
0810 14:44:51.642427 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 14:44:51.642699 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:51.634969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:50745 (local address 127.24.215.131:42689)
0810 14:44:51.635244 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:51.635253 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:51.636102 (+ 849us) server_negotiation.cc:407] Connection header received
0810 14:44:51.636259 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:51.636263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:51.636319 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:51.636428 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:44:51.636960 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.637654 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.640114 (+ 2460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.641436 (+ 1322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.641599 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:51.641671 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:51.642292 (+ 621us) server_negotiation.cc:299] Negotiation successful
0810 14:44:51.642427 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 14:44:51.642902 30976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:51.635033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:42689 (local address 127.24.215.129:50745)
0810 14:44:51.635991 (+ 958us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:51.636003 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:51.636137 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:51.636438 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:51.636442 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:51.636469 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:51.636802 (+ 333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:51.636810 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.637814 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.637820 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:51.639922 (+ 2102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:51.639934 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.641692 (+ 1758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.641699 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:51.642027 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:51.642146 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 14:44:51.642246 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 14:44:51.642384 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"mutex_wait_us":142,"thread_start_us":689,"threads_started":1}
I20260810 14:44:51.642937 30974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:51.634812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:34781 (local address 127.24.215.129:55709)
0810 14:44:51.635447 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:51.635463 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:51.635578 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:51.636132 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:51.636136 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:51.636162 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:44:51.636496 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:51.636505 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.637501 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.637509 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:51.639918 (+ 2409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:51.639934 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.641706 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.641711 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:51.642028 (+ 317us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:51.642147 (+ 119us) client_negotiation.cc:769] Sending connection context
0810 14:44:51.642246 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 14:44:51.642395 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":115,"thread_start_us":434,"threads_started":1}
I20260810 14:44:51.644187 30887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462f3f141aea44b48c27dcdaf82b75e1" candidate_uuid: "ea2fb8554a944145ac5d2faa2a0414ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4b55ae8f1514665a1c54aabb03d15c4" is_pre_election: true
I20260810 14:44:51.644187 30716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462f3f141aea44b48c27dcdaf82b75e1" candidate_uuid: "ea2fb8554a944145ac5d2faa2a0414ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f2c763f57c641e5bebf76078a61f9a0" is_pre_election: true
I20260810 14:44:51.644824 30716 raft_consensus.cc:2454] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea2fb8554a944145ac5d2faa2a0414ed in term 0.
I20260810 14:44:51.644824 30887 raft_consensus.cc:2454] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea2fb8554a944145ac5d2faa2a0414ed in term 0.
I20260810 14:44:51.645627 30477 leader_election.cc:302] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea2fb8554a944145ac5d2faa2a0414ed, f4b55ae8f1514665a1c54aabb03d15c4; no voters:
I20260810 14:44:51.646112 30972 raft_consensus.cc:2781] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:44:51.646289 30972 raft_consensus.cc:482] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:44:51.646425 30972 raft_consensus.cc:3037] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:51.648761 30972 raft_consensus.cc:504] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b55ae8f1514665a1c54aabb03d15c4" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 42689 } } peers { permanent_uuid: "ea2fb8554a944145ac5d2faa2a0414ed" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44843 } } peers { permanent_uuid: "2f2c763f57c641e5bebf76078a61f9a0" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 34781 } }
I20260810 14:44:51.649369 30972 leader_election.cc:288] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [CANDIDATE]: Term 1 election: Requested vote from peers f4b55ae8f1514665a1c54aabb03d15c4 (127.24.215.131:42689), 2f2c763f57c641e5bebf76078a61f9a0 (127.24.215.130:34781)
I20260810 14:44:51.649935 30887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462f3f141aea44b48c27dcdaf82b75e1" candidate_uuid: "ea2fb8554a944145ac5d2faa2a0414ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4b55ae8f1514665a1c54aabb03d15c4"
I20260810 14:44:51.649945 30716 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "462f3f141aea44b48c27dcdaf82b75e1" candidate_uuid: "ea2fb8554a944145ac5d2faa2a0414ed" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f2c763f57c641e5bebf76078a61f9a0"
I20260810 14:44:51.650164 30716 raft_consensus.cc:3037] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:51.650164 30887 raft_consensus.cc:3037] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:51.653486 30887 raft_consensus.cc:2454] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea2fb8554a944145ac5d2faa2a0414ed in term 1.
I20260810 14:44:51.653486 30716 raft_consensus.cc:2454] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea2fb8554a944145ac5d2faa2a0414ed in term 1.
I20260810 14:44:51.653959 30477 leader_election.cc:302] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ea2fb8554a944145ac5d2faa2a0414ed, f4b55ae8f1514665a1c54aabb03d15c4; no voters:
I20260810 14:44:51.654286 30972 raft_consensus.cc:2781] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:51.654742 30972 raft_consensus.cc:686] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [term 1 LEADER]: Becoming Leader. State: Replica: ea2fb8554a944145ac5d2faa2a0414ed, State: Running, Role: LEADER
I20260810 14:44:51.655303 30972 consensus_queue.cc:237] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b55ae8f1514665a1c54aabb03d15c4" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 42689 } } peers { permanent_uuid: "ea2fb8554a944145ac5d2faa2a0414ed" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44843 } } peers { permanent_uuid: "2f2c763f57c641e5bebf76078a61f9a0" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 34781 } }
I20260810 14:44:51.660163 30382 catalog_manager.cc:5495] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed reported cstate change: term changed from 0 to 1, leader changed from <none> to ea2fb8554a944145ac5d2faa2a0414ed (127.24.215.129). New cstate: current_term: 1 leader_uuid: "ea2fb8554a944145ac5d2faa2a0414ed" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f4b55ae8f1514665a1c54aabb03d15c4" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 42689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea2fb8554a944145ac5d2faa2a0414ed" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44843 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f2c763f57c641e5bebf76078a61f9a0" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 34781 } health_report { overall_health: UNKNOWN } } }
I20260810 14:44:51.678716 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:51.673259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35936 (local address 127.24.215.129:44843)
0810 14:44:51.673484 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:51.673491 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:51.673560 (+ 69us) server_negotiation.cc:407] Connection header received
0810 14:44:51.673813 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:51.673816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:51.673903 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:51.674071 (+ 168us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:44:51.674475 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.675256 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.676612 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.677045 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.677177 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:51.677649 (+ 472us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:44:51.678143 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:44:51.678308 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:51.678365 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 14:44:51.678447 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 14:44:51.686995 30716 raft_consensus.cc:1261] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0 [term 1 FOLLOWER]: Refusing update from remote peer ea2fb8554a944145ac5d2faa2a0414ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:51.686987 30887 raft_consensus.cc:1261] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4 [term 1 FOLLOWER]: Refusing update from remote peer ea2fb8554a944145ac5d2faa2a0414ed: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:51.688084 30977 consensus_queue.cc:1035] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4b55ae8f1514665a1c54aabb03d15c4" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 42689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:51.688450 30972 consensus_queue.cc:1035] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f2c763f57c641e5bebf76078a61f9a0" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 34781 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:51.717212 30385 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:44:51.717550 30385 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42286:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:44:51.722131 30385 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 37b7ed0145d54177835057ce932fedad: Sending DeleteTablet for 3 replicas of tablet 462f3f141aea44b48c27dcdaf82b75e1
I20260810 14:44:51.723253 30861 tablet_service.cc:1514] Processing DeleteTablet for tablet 462f3f141aea44b48c27dcdaf82b75e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:51 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:60684
I20260810 14:44:51.723904 30860 tablet_service.cc:1514] Processing DeleteTablet for tablet 462f3f141aea44b48c27dcdaf82b75e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:51 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:60684
I20260810 14:44:51.724030 30530 tablet_service.cc:1514] Processing DeleteTablet for tablet 462f3f141aea44b48c27dcdaf82b75e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:51 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:35926
I20260810 14:44:51.724118 30691 tablet_service.cc:1514] Processing DeleteTablet for tablet 462f3f141aea44b48c27dcdaf82b75e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:51 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:45020
I20260810 14:44:51.726235 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 30455
I20260810 14:44:51.732496 30994 tablet_replica.cc:330] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed: stopping tablet replica
I20260810 14:44:51.733072 30994 raft_consensus.cc:2229] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:44:51.733914 30994 raft_consensus.cc:2258] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:51.734833 30995 tablet_replica.cc:330] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0: stopping tablet replica
I20260810 14:44:51.735343 30995 raft_consensus.cc:2229] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:44:51.737219 30993 tablet_replica.cc:330] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4: stopping tablet replica
I20260810 14:44:51.737710 30993 raft_consensus.cc:2229] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:44:51.737771 30994 ts_tablet_manager.cc:1905] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:51.738289 30993 raft_consensus.cc:2258] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:51.740537 30993 ts_tablet_manager.cc:1905] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:51.740757 30995 raft_consensus.cc:2258] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 10 14:44:51 dist-test-slave-vkjp krb5kdc[30328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:44:51.745316 30995 ts_tablet_manager.cc:1905] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:51.748348 30993 ts_tablet_manager.cc:1918] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:44:51.748602 30993 log.cc:1196] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/462f3f141aea44b48c27dcdaf82b75e1
I20260810 14:44:51.749233 30993 ts_tablet_manager.cc:1939] T 462f3f141aea44b48c27dcdaf82b75e1 P f4b55ae8f1514665a1c54aabb03d15c4: Deleting consensus metadata
I20260810 14:44:51.751300 30364 catalog_manager.cc:4841] TS f4b55ae8f1514665a1c54aabb03d15c4 (127.24.215.131:42689): tablet 462f3f141aea44b48c27dcdaf82b75e1 (table test-table [id=92d0b033167840268538b6edea65da8b]) successfully deleted
W20260810 14:44:51.752401 30364 catalog_manager.cc:4808] TS f4b55ae8f1514665a1c54aabb03d15c4 (127.24.215.131:42689): delete failed for tablet 462f3f141aea44b48c27dcdaf82b75e1 because the tablet was not found. No further retry: Not found: Tablet not found: 462f3f141aea44b48c27dcdaf82b75e1
I20260810 14:44:51.752722 30994 ts_tablet_manager.cc:1918] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:44:51.752910 30994 log.cc:1196] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/462f3f141aea44b48c27dcdaf82b75e1
I20260810 14:44:51.752944 30995 ts_tablet_manager.cc:1918] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:44:51.753162 30995 log.cc:1196] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/462f3f141aea44b48c27dcdaf82b75e1
I20260810 14:44:51.753532 30994 ts_tablet_manager.cc:1939] T 462f3f141aea44b48c27dcdaf82b75e1 P ea2fb8554a944145ac5d2faa2a0414ed: Deleting consensus metadata
I20260810 14:44:51.753801 30995 ts_tablet_manager.cc:1939] T 462f3f141aea44b48c27dcdaf82b75e1 P 2f2c763f57c641e5bebf76078a61f9a0: Deleting consensus metadata
I20260810 14:44:51.755332 30362 catalog_manager.cc:4841] TS ea2fb8554a944145ac5d2faa2a0414ed (127.24.215.129:44843): tablet 462f3f141aea44b48c27dcdaf82b75e1 (table test-table [id=92d0b033167840268538b6edea65da8b]) successfully deleted
I20260810 14:44:51.755733 30363 catalog_manager.cc:4841] TS 2f2c763f57c641e5bebf76078a61f9a0 (127.24.215.130:34781): tablet 462f3f141aea44b48c27dcdaf82b75e1 (table test-table [id=92d0b033167840268538b6edea65da8b]) successfully deleted
I20260810 14:44:51.756673 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:51.728679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35948 (local address 127.24.215.129:44843)
0810 14:44:51.728897 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:51.728906 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:51.732111 (+ 3205us) server_negotiation.cc:407] Connection header received
0810 14:44:51.733016 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:51.733022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:51.733140 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:51.733226 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:51.735165 (+ 1939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.736409 (+ 1244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.738298 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.739105 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.739227 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:51.744830 (+ 5603us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:51.744845 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:51.744862 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:51.744920 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:51.749052 (+ 4132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:51.753843 (+ 4791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:51.753856 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:51.753860 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:51.754092 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:51.754607 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:51.754612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:51.754615 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:51.755133 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:51.755431 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:51.756065 (+ 634us) server_negotiation.cc:299] Negotiation successful
0810 14:44:51.756310 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:44:51.757280 30609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:51.856237 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 30455
I20260810 14:44:51.879662 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 30622
Aug 10 14:44:51 dist-test-slave-vkjp krb5kdc[30328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:44:51.895336 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:51.880487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45024 (local address 127.24.215.130:34781)
0810 14:44:51.880666 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:51.880670 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:51.880688 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:44:51.880830 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:51.880834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:51.880909 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:51.880994 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:51.882177 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.882916 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.884217 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:51.884735 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:51.884879 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:51.889129 (+ 4250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:51.889145 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:51.889162 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:51.889204 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:51.892106 (+ 2902us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:51.892854 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:51.892862 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:51.892865 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:51.893012 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:51.893444 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:51.893451 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:51.893455 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:51.893869 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:51.894205 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:51.894549 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 14:44:51.894799 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:44:51.896098 30767 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:51.986491 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 30622
I20260810 14:44:52.008499 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 30782
Aug 10 14:44:52 dist-test-slave-vkjp krb5kdc[30328](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:44:52.023823 30955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:52.009294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60690 (local address 127.24.215.131:42689)
0810 14:44:52.009486 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:52.009490 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:52.009509 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:44:52.009642 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:52.009646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:52.009731 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:52.009816 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:52.011172 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:52.011924 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:52.013161 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:52.013619 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:52.013723 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:52.018200 (+ 4477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:52.018216 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:52.018234 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:44:52.018273 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:52.021195 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:52.021823 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:52.021831 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:52.021833 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:52.021968 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:44:52.022329 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:44:52.022332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:44:52.022334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:44:52.022691 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:52.022907 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:52.023251 (+ 344us) server_negotiation.cc:299] Negotiation successful
0810 14:44:52.023498 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 14:44:52.024482 30935 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:52.113543 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 30782
I20260810 14:44:52.135586 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 30344
I20260810 14:44:52.136770 30423 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:52.245553 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 30344
2026-08-10T14:44:52Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4386 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-10T14:44:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:44:52Z Disabled control of system clock
I20260810 14:44:52.322001 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:46289
--webserver_interface=127.24.215.190
--webserver_port=0
--builtin_ntp_servers=127.24.215.148:44347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:46289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 14:44:52.657616 31005 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:52.657917 31005 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:52.658003 31005 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:52.665165 31005 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:52.665277 31005 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:52.665318 31005 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:52.665382 31005 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:52.665428 31005 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:44:52.674453 31005 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:46289
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:46289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:52.675732 31005 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:52.677510 31005 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:52.689571 31010 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:52.689492 31011 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:52.689524 31014 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:52.690456 31005 server_base.cc:1034] running on GCE node
I20260810 14:44:52.691357 31005 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:52.692823 31005 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:52.694063 31005 hybrid_clock.cc:648] HybridClock initialized: now 1786373092694034 us; error 55 us; skew 500 ppm
I20260810 14:44:52.694410 31005 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:44:52.696763 31005 webserver.cc:460] Webserver started at http://127.24.215.190:44641/ using document root <none> and password file <none>
I20260810 14:44:52.697482 31005 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:52.697621 31005 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:52.697965 31005 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:52.700937 31005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "eb9cf27daec24089bb31bedfd5409adb"
format_stamp: "Formatted at 2026-08-10 14:44:52 on dist-test-slave-vkjp"
server_key: "72989faf19e33fa085e658dba47db28d"
server_key_iv: "f40758404a4433f1120c2d49d8477ac0"
server_key_version: "encryptionkey@0"
I20260810 14:44:52.701772 31005 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "eb9cf27daec24089bb31bedfd5409adb"
format_stamp: "Formatted at 2026-08-10 14:44:52 on dist-test-slave-vkjp"
server_key: "72989faf19e33fa085e658dba47db28d"
server_key_iv: "f40758404a4433f1120c2d49d8477ac0"
server_key_version: "encryptionkey@0"
I20260810 14:44:52.706956 31005 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260810 14:44:52.710470 31023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:52.711923 31005 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:44:52.712119 31005 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "eb9cf27daec24089bb31bedfd5409adb"
format_stamp: "Formatted at 2026-08-10 14:44:52 on dist-test-slave-vkjp"
server_key: "72989faf19e33fa085e658dba47db28d"
server_key_iv: "f40758404a4433f1120c2d49d8477ac0"
server_key_version: "encryptionkey@0"
I20260810 14:44:52.712352 31005 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:52.718055 31005 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:52.718825 31005 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:52.719183 31005 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:52.737275 31005 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:46289
I20260810 14:44:52.737311 31082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:46289 every 8 connection(s)
I20260810 14:44:52.738598 31005 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:44:52.742376 31083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:52.742692 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 31005
I20260810 14:44:52.742893 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:44:52.743763 25438 external_mini_cluster.cc:1428] Setting key 58b2b58533c9158aafcc72f18e5798a7
I20260810 14:44:52.752449 31086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:52.745874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48972 (local address 127.24.215.190:46289)
0810 14:44:52.746937 (+ 1063us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:52.746948 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:52.746999 (+ 51us) server_negotiation.cc:407] Connection header received
0810 14:44:52.748878 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:52.748925 (+ 47us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:52.749354 (+ 429us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:52.749679 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:52.750837 (+ 1158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:52.750844 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:52.750856 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:52.750870 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:52.750942 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:52.750958 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:52.751196 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:52.751297 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 14:44:52.751555 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":390,"threads_started":1}
I20260810 14:44:52.756345 31083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb: Bootstrap starting.
I20260810 14:44:52.759490 31083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:52.760484 31083 log.cc:824] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:52.763322 31083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb: No bootstrap required, opened a new log
I20260810 14:44:52.770545 31083 raft_consensus.cc:348] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9cf27daec24089bb31bedfd5409adb" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46289 } }
I20260810 14:44:52.770915 31083 raft_consensus.cc:374] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:52.771039 31083 raft_consensus.cc:729] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb9cf27daec24089bb31bedfd5409adb, State: Initialized, Role: FOLLOWER
I20260810 14:44:52.771636 31083 consensus_queue.cc:260] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9cf27daec24089bb31bedfd5409adb" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46289 } }
I20260810 14:44:52.771864 31083 raft_consensus.cc:388] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:44:52.772009 31083 raft_consensus.cc:482] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:44:52.772185 31083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:52.774320 31083 raft_consensus.cc:504] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9cf27daec24089bb31bedfd5409adb" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46289 } }
I20260810 14:44:52.774837 31083 leader_election.cc:302] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: eb9cf27daec24089bb31bedfd5409adb; no voters:
I20260810 14:44:52.775592 31083 leader_election.cc:288] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:44:52.775702 31088 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:52.776722 31088 raft_consensus.cc:686] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [term 1 LEADER]: Becoming Leader. State: Replica: eb9cf27daec24089bb31bedfd5409adb, State: Running, Role: LEADER
I20260810 14:44:52.777298 31088 consensus_queue.cc:237] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9cf27daec24089bb31bedfd5409adb" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46289 } }
I20260810 14:44:52.778134 31083 sys_catalog.cc:564] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:44:52.780579 31090 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [sys.catalog]: SysCatalogTable state changed. Reason: New leader eb9cf27daec24089bb31bedfd5409adb. Latest consensus state: current_term: 1 leader_uuid: "eb9cf27daec24089bb31bedfd5409adb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9cf27daec24089bb31bedfd5409adb" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46289 } } }
I20260810 14:44:52.780695 31089 sys_catalog.cc:455] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "eb9cf27daec24089bb31bedfd5409adb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "eb9cf27daec24089bb31bedfd5409adb" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46289 } } }
I20260810 14:44:52.781109 31090 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:52.781193 31089 sys_catalog.cc:458] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:52.782799 31095 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:44:52.788728 31095 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:44:52.797724 31095 catalog_manager.cc:1347] Generated new cluster ID: 76c5ad7b61b140f3b773643c970ea1bb
I20260810 14:44:52.797868 31095 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:44:52.822517 31095 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:44:52.822956 31095 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:44:52.833618 31095 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb: Generated new TSK 0
I20260810 14:44:52.834681 31095 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:44:52.852485 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46289
--builtin_ntp_servers=127.24.215.148:44347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260810 14:44:53.184835 31110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:53.185113 31110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:53.185205 31110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:53.192400 31110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:53.192503 31110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:53.192613 31110 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:44:53.201648 31110 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46289
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:53.202915 31110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:53.204764 31110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:53.218036 31116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:53.218117 31121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:53.218175 31118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:53.219044 31110 server_base.cc:1034] running on GCE node
I20260810 14:44:53.219720 31110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:53.220646 31110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:53.221879 31110 hybrid_clock.cc:648] HybridClock initialized: now 1786373093221833 us; error 48 us; skew 500 ppm
I20260810 14:44:53.222200 31110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:44:53.224594 31110 webserver.cc:460] Webserver started at http://127.24.215.129:34505/ using document root <none> and password file <none>
I20260810 14:44:53.225303 31110 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:53.225440 31110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:53.225782 31110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:53.228662 31110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "4120b838370e43dabfe66376b80b1dc6"
format_stamp: "Formatted at 2026-08-10 14:44:53 on dist-test-slave-vkjp"
server_key: "5f1ae7d8dd0b1f825412064fa050eb78"
server_key_iv: "89aa714639fafcfb2fc8672c2c4cfb92"
server_key_version: "encryptionkey@0"
I20260810 14:44:53.229475 31110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "4120b838370e43dabfe66376b80b1dc6"
format_stamp: "Formatted at 2026-08-10 14:44:53 on dist-test-slave-vkjp"
server_key: "5f1ae7d8dd0b1f825412064fa050eb78"
server_key_iv: "89aa714639fafcfb2fc8672c2c4cfb92"
server_key_version: "encryptionkey@0"
I20260810 14:44:53.234501 31110 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 14:44:53.237870 31130 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:53.239049 31110 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 14:44:53.239281 31110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "4120b838370e43dabfe66376b80b1dc6"
format_stamp: "Formatted at 2026-08-10 14:44:53 on dist-test-slave-vkjp"
server_key: "5f1ae7d8dd0b1f825412064fa050eb78"
server_key_iv: "89aa714639fafcfb2fc8672c2c4cfb92"
server_key_version: "encryptionkey@0"
I20260810 14:44:53.239498 31110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:53.244894 31110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:53.245656 31110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:53.245949 31110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:53.246827 31110 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:53.248299 31110 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:53.248423 31110 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:53.248565 31110 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:53.248656 31110 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:53.281816 31110 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:40327
I20260810 14:44:53.281944 31268 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:40327 every 8 connection(s)
I20260810 14:44:53.283303 31110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:44:53.284145 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 31110
I20260810 14:44:53.284351 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:44:53.285228 25438 external_mini_cluster.cc:1428] Setting key 7530cdf2f72135a87e382c658a7ac152
I20260810 14:44:53.288707 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46289
--builtin_ntp_servers=127.24.215.148:44347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 14:44:53.293936 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:53.286632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:54975 (local address 127.24.215.190:46289)
0810 14:44:53.287139 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:53.287145 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:53.288774 (+ 1629us) server_negotiation.cc:407] Connection header received
0810 14:44:53.291586 (+ 2812us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:53.291590 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:53.291643 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:53.291790 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:53.293171 (+ 1381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:53.293176 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:53.293187 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:53.293204 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:53.293244 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:53.293256 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:53.293387 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:53.293588 (+ 201us) server_negotiation.cc:299] Negotiation successful
0810 14:44:53.293656 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":238,"threads_started":1}
I20260810 14:44:53.294948 31273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:53.287409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:46289 (local address 127.24.215.129:54975)
0810 14:44:53.288587 (+ 1178us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:53.288651 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:53.291281 (+ 2630us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:53.292055 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:53.292063 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:53.292155 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:53.292857 (+ 702us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:44:53.292859 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:53.292898 (+ 39us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:44:53.292992 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:53.293353 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:53.293366 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:53.293374 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 14:44:53.293671 (+ 297us) client_negotiation.cc:240] Negotiation successful
0810 14:44:53.293933 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":332,"threads_started":1}
I20260810 14:44:53.296525 31269 heartbeater.cc:347] Connected to a master server at 127.24.215.190:46289
I20260810 14:44:53.296875 31269 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:53.297664 31269 heartbeater.cc:511] Master 127.24.215.190:46289 requested a full tablet report, sending...
I20260810 14:44:53.299863 31044 ts_manager.cc:194] Registered new tserver with Master: 4120b838370e43dabfe66376b80b1dc6 (127.24.215.129:40327)
W20260810 14:44:53.629050 31274 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:53.629390 31274 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:53.629557 31274 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:53.636902 31274 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:53.637058 31274 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:53.637246 31274 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:44:53.646589 31274 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46289
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:53.648056 31274 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:53.649863 31274 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:53.663424 31284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:53.663492 31281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:53.663381 31286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:53.668733 31274 server_base.cc:1034] running on GCE node
I20260810 14:44:53.669432 31274 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:53.670447 31274 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:53.671690 31274 hybrid_clock.cc:648] HybridClock initialized: now 1786373093671644 us; error 41 us; skew 500 ppm
I20260810 14:44:53.672048 31274 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:44:53.674523 31274 webserver.cc:460] Webserver started at http://127.24.215.130:42973/ using document root <none> and password file <none>
I20260810 14:44:53.675386 31274 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:53.675524 31274 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:53.675874 31274 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:53.679032 31274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "9095ed6a918c4ee483876b42b95f6444"
format_stamp: "Formatted at 2026-08-10 14:44:53 on dist-test-slave-vkjp"
server_key: "599b16442151cf0585bd84a29a123933"
server_key_iv: "2107a9cc7cdbf228bb53fac5c71f64ac"
server_key_version: "encryptionkey@0"
I20260810 14:44:53.679998 31274 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "9095ed6a918c4ee483876b42b95f6444"
format_stamp: "Formatted at 2026-08-10 14:44:53 on dist-test-slave-vkjp"
server_key: "599b16442151cf0585bd84a29a123933"
server_key_iv: "2107a9cc7cdbf228bb53fac5c71f64ac"
server_key_version: "encryptionkey@0"
I20260810 14:44:53.685657 31274 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 14:44:53.689266 31295 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:53.690626 31274 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:44:53.690816 31274 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "9095ed6a918c4ee483876b42b95f6444"
format_stamp: "Formatted at 2026-08-10 14:44:53 on dist-test-slave-vkjp"
server_key: "599b16442151cf0585bd84a29a123933"
server_key_iv: "2107a9cc7cdbf228bb53fac5c71f64ac"
server_key_version: "encryptionkey@0"
I20260810 14:44:53.691061 31274 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:53.697165 31274 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:53.698004 31274 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:53.698285 31274 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:53.699338 31274 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:53.700770 31274 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:53.700904 31274 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:53.701056 31274 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:53.701148 31274 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:53.735384 31274 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:36431
I20260810 14:44:53.735476 31434 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:36431 every 8 connection(s)
I20260810 14:44:53.736771 31274 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:44:53.739359 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 31274
I20260810 14:44:53.739519 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:44:53.740025 25438 external_mini_cluster.cc:1428] Setting key 73b13c6e0b7be52faf97ae88b0381319
I20260810 14:44:53.743299 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46289
--builtin_ntp_servers=127.24.215.148:44347
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260810 14:44:53.746398 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:53.739650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:39091 (local address 127.24.215.190:46289)
0810 14:44:53.739874 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:53.739880 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:53.741427 (+ 1547us) server_negotiation.cc:407] Connection header received
0810 14:44:53.743814 (+ 2387us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:53.743820 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:53.743903 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:53.744004 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:53.745624 (+ 1620us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:53.745630 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:53.745635 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:53.745656 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:53.745696 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:53.745711 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:53.745833 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:53.746050 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 14:44:53.746110 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 14:44:53.747639 31437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:53.740171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:46289 (local address 127.24.215.130:39091)
0810 14:44:53.741254 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:53.741298 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:53.743545 (+ 2247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:53.744253 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:53.744267 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:53.744392 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:53.745252 (+ 860us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:44:53.745257 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:53.745312 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:44:53.745435 (+ 123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:53.745822 (+ 387us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:53.745835 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:53.745845 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:44:53.746122 (+ 277us) client_negotiation.cc:240] Negotiation successful
0810 14:44:53.746375 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":352,"threads_started":1}
I20260810 14:44:53.749169 31435 heartbeater.cc:347] Connected to a master server at 127.24.215.190:46289
I20260810 14:44:53.749496 31435 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:53.750167 31435 heartbeater.cc:511] Master 127.24.215.190:46289 requested a full tablet report, sending...
I20260810 14:44:53.751946 31044 ts_manager.cc:194] Registered new tserver with Master: 9095ed6a918c4ee483876b42b95f6444 (127.24.215.130:36431)
W20260810 14:44:54.090638 31439 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:54.090965 31439 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:54.091173 31439 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:54.098835 31439 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:54.098979 31439 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:54.099208 31439 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:44:54.108487 31439 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44347
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46289
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:54.109896 31439 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:54.111771 31439 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:54.125198 31446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:54.125227 31447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:54.125138 31449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:54.126323 31439 server_base.cc:1034] running on GCE node
I20260810 14:44:54.127041 31439 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:54.128062 31439 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:54.129271 31439 hybrid_clock.cc:648] HybridClock initialized: now 1786373094129230 us; error 28 us; skew 500 ppm
I20260810 14:44:54.129613 31439 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:44:54.132047 31439 webserver.cc:460] Webserver started at http://127.24.215.131:38091/ using document root <none> and password file <none>
I20260810 14:44:54.132778 31439 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:54.132889 31439 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:54.133261 31439 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:54.136231 31439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "33ae524031954fc69f9d737eeb2a9408"
format_stamp: "Formatted at 2026-08-10 14:44:54 on dist-test-slave-vkjp"
server_key: "a6d98f468000f19c582541e7363806ed"
server_key_iv: "169fa976acf079e7e024d2fc02a76c76"
server_key_version: "encryptionkey@0"
I20260810 14:44:54.137040 31439 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "33ae524031954fc69f9d737eeb2a9408"
format_stamp: "Formatted at 2026-08-10 14:44:54 on dist-test-slave-vkjp"
server_key: "a6d98f468000f19c582541e7363806ed"
server_key_iv: "169fa976acf079e7e024d2fc02a76c76"
server_key_version: "encryptionkey@0"
I20260810 14:44:54.142127 31439 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260810 14:44:54.145700 31457 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:54.147018 31439 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:44:54.147294 31439 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "33ae524031954fc69f9d737eeb2a9408"
format_stamp: "Formatted at 2026-08-10 14:44:54 on dist-test-slave-vkjp"
server_key: "a6d98f468000f19c582541e7363806ed"
server_key_iv: "169fa976acf079e7e024d2fc02a76c76"
server_key_version: "encryptionkey@0"
I20260810 14:44:54.147532 31439 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:54.153168 31439 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:54.153890 31439 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:54.154109 31439 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:54.154956 31439 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:44:54.156386 31439 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:44:54.156481 31439 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:54.156562 31439 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:44:54.156617 31439 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:54.189872 31439 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:43921
I20260810 14:44:54.189926 31592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:43921 every 8 connection(s)
I20260810 14:44:54.191283 31439 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:44:54.199082 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 31439
I20260810 14:44:54.199302 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:44:54.199900 25438 external_mini_cluster.cc:1428] Setting key 8cf3a56caa2adbb6720f6bcd1c122cc7
I20260810 14:44:54.200006 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.193971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:50039 (local address 127.24.215.190:46289)
0810 14:44:54.194185 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:54.194191 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:54.195791 (+ 1600us) server_negotiation.cc:407] Connection header received
0810 14:44:54.197524 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:54.197529 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:54.197588 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:54.197689 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:54.199156 (+ 1467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:54.199162 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:54.199167 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:54.199189 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:54.199240 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:54.199258 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:54.199397 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:54.199619 (+ 222us) server_negotiation.cc:299] Negotiation successful
0810 14:44:54.199690 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 14:44:54.200949 31595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.194562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:46289 (local address 127.24.215.131:50039)
0810 14:44:54.195581 (+ 1019us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:54.195621 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:54.197240 (+ 1619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:54.197937 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:54.197945 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:54.198035 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:54.198808 (+ 773us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:44:54.198811 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:54.198851 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:44:54.198942 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:54.199378 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:54.199393 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:54.199401 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 14:44:54.199642 (+ 241us) client_negotiation.cc:240] Negotiation successful
0810 14:44:54.199846 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":307,"threads_started":1}
I20260810 14:44:54.202493 31593 heartbeater.cc:347] Connected to a master server at 127.24.215.190:46289
I20260810 14:44:54.202854 31593 heartbeater.cc:464] Registering TS with master...
I20260810 14:44:54.203573 31593 heartbeater.cc:511] Master 127.24.215.190:46289 requested a full tablet report, sending...
I20260810 14:44:54.204875 31044 ts_manager.cc:194] Registered new tserver with Master: 33ae524031954fc69f9d737eeb2a9408 (127.24.215.131:43921)
I20260810 14:44:54.215493 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:44:54.222296 31271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.220050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48984 (local address 127.24.215.190:46289)
0810 14:44:54.220225 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:54.220230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:54.220750 (+ 520us) server_negotiation.cc:407] Connection header received
0810 14:44:54.220883 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:54.220886 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:54.220937 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:54.221030 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:54.221866 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:54.221871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:54.221874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:54.221888 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:54.221918 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:54.221930 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:54.222023 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:54.222057 (+ 34us) server_negotiation.cc:299] Negotiation successful
0810 14:44:54.222099 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 14:44:54.226357 31044 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48984:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:44:54.233121 31044 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:44:54.259599 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.245377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60850 (local address 127.24.215.129:40327)
0810 14:44:54.256016 (+ 10639us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:54.256027 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:54.256060 (+ 33us) server_negotiation.cc:407] Connection header received
0810 14:44:54.256137 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:54.256146 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:54.256393 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:54.257214 (+ 821us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:54.258639 (+ 1425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:54.258651 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:54.258669 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:54.258688 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:54.258742 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:54.258758 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:54.258915 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:54.259102 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 14:44:54.259207 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10454,"thread_start_us":10289,"threads_started":1}
I20260810 14:44:54.259666 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.244972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:40327 (local address 127.0.0.1:60850)
0810 14:44:54.255380 (+ 10408us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:54.255436 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:54.255564 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:54.256606 (+ 1042us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:54.256613 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:54.256702 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:54.257638 (+ 936us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:44:54.257641 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:54.257677 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:44:54.257768 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:54.258867 (+ 1099us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:54.258878 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:54.258886 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 14:44:54.259128 (+ 242us) client_negotiation.cc:240] Negotiation successful
0810 14:44:54.259214 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10169,"thread_start_us":10827,"threads_started":1}
I20260810 14:44:54.261144 31606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.245525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:43921 (local address 127.0.0.1:48280)
0810 14:44:54.256057 (+ 10532us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:54.256075 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:54.256241 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:54.257934 (+ 1693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:54.257938 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:54.257960 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:54.260109 (+ 2149us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:44:54.260113 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:54.260137 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:44:54.260174 (+ 37us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:54.260546 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:54.260551 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:54.260555 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:44:54.260688 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 14:44:54.260739 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10404,"thread_start_us":9424,"threads_started":1}
I20260810 14:44:54.261209 31609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.246119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48280 (local address 127.24.215.131:43921)
0810 14:44:54.246848 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:54.246861 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:54.256163 (+ 9302us) server_negotiation.cc:407] Connection header received
0810 14:44:54.256379 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:54.256386 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:54.256592 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:54.256707 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:54.260333 (+ 3626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:54.260341 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:54.260354 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:54.260377 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:54.260425 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:54.260440 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:54.260627 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:54.260694 (+ 67us) server_negotiation.cc:299] Negotiation successful
0810 14:44:54.260785 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":523,"thread_start_us":379,"threads_started":1}
I20260810 14:44:54.263192 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.256203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:36431 (local address 127.0.0.1:42460)
0810 14:44:54.259838 (+ 3635us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:54.259853 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:54.259923 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:54.261560 (+ 1637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:54.261565 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:54.261616 (+ 51us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:54.262149 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:44:54.262152 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:54.262167 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:44:54.262209 (+ 42us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:54.262719 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:54.262724 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:54.262729 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:44:54.262831 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 14:44:54.262877 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3550,"thread_start_us":4155,"threads_started":1}
I20260810 14:44:54.263965 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.256555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42460 (local address 127.24.215.130:36431)
0810 14:44:54.260166 (+ 3611us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:54.260176 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:54.260207 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:44:54.260287 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:54.260296 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:54.260507 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:54.260666 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:54.262383 (+ 1717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:54.262392 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:54.262404 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:54.262436 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:54.262496 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:54.262519 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:54.262660 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:54.263548 (+ 888us) server_negotiation.cc:299] Negotiation successful
0810 14:44:54.263642 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3441,"thread_start_us":3321,"threads_started":1}
I20260810 14:44:54.264570 31190 tablet_service.cc:1467] Processing CreateTablet for tablet b64a89918c8e419da4be9c2c680946c9 (DEFAULT_TABLE table=test-table [id=861eaa536fa047fea9d85395a840461f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:54.264922 31513 tablet_service.cc:1467] Processing CreateTablet for tablet b64a89918c8e419da4be9c2c680946c9 (DEFAULT_TABLE table=test-table [id=861eaa536fa047fea9d85395a840461f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:54.265806 31190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b64a89918c8e419da4be9c2c680946c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:54.266330 31513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b64a89918c8e419da4be9c2c680946c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:54.267449 31352 tablet_service.cc:1467] Processing CreateTablet for tablet b64a89918c8e419da4be9c2c680946c9 (DEFAULT_TABLE table=test-table [id=861eaa536fa047fea9d85395a840461f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:44:54.268754 31352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b64a89918c8e419da4be9c2c680946c9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:54.277144 31612 tablet_bootstrap.cc:492] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6: Bootstrap starting.
I20260810 14:44:54.280642 31612 tablet_bootstrap.cc:654] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:54.280882 31613 tablet_bootstrap.cc:492] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408: Bootstrap starting.
I20260810 14:44:54.282176 31612 log.cc:824] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:54.284734 31613 tablet_bootstrap.cc:654] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:54.286104 31613 log.cc:824] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:54.286348 31612 tablet_bootstrap.cc:492] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6: No bootstrap required, opened a new log
I20260810 14:44:54.286700 31612 ts_tablet_manager.cc:1397] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260810 14:44:54.288558 31616 tablet_bootstrap.cc:492] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444: Bootstrap starting.
I20260810 14:44:54.292119 31616 tablet_bootstrap.cc:654] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:54.293352 31616 log.cc:824] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:54.295486 31613 tablet_bootstrap.cc:492] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408: No bootstrap required, opened a new log
I20260810 14:44:54.295840 31613 ts_tablet_manager.cc:1397] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408: Time spent bootstrapping tablet: real 0.015s user 0.010s sys 0.004s
I20260810 14:44:54.300868 31616 tablet_bootstrap.cc:492] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444: No bootstrap required, opened a new log
I20260810 14:44:54.301160 31616 ts_tablet_manager.cc:1397] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444: Time spent bootstrapping tablet: real 0.013s user 0.010s sys 0.000s
I20260810 14:44:54.301676 31612 raft_consensus.cc:348] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4120b838370e43dabfe66376b80b1dc6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 40327 } } peers { permanent_uuid: "9095ed6a918c4ee483876b42b95f6444" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36431 } } peers { permanent_uuid: "33ae524031954fc69f9d737eeb2a9408" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 43921 } }
I20260810 14:44:54.302244 31612 raft_consensus.cc:374] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:54.302366 31612 raft_consensus.cc:729] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4120b838370e43dabfe66376b80b1dc6, State: Initialized, Role: FOLLOWER
I20260810 14:44:54.303074 31269 heartbeater.cc:503] Master 127.24.215.190:46289 was elected leader, sending a full tablet report...
I20260810 14:44:54.303009 31612 consensus_queue.cc:260] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4120b838370e43dabfe66376b80b1dc6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 40327 } } peers { permanent_uuid: "9095ed6a918c4ee483876b42b95f6444" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36431 } } peers { permanent_uuid: "33ae524031954fc69f9d737eeb2a9408" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 43921 } }
I20260810 14:44:54.307518 31613 raft_consensus.cc:348] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4120b838370e43dabfe66376b80b1dc6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 40327 } } peers { permanent_uuid: "9095ed6a918c4ee483876b42b95f6444" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36431 } } peers { permanent_uuid: "33ae524031954fc69f9d737eeb2a9408" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 43921 } }
I20260810 14:44:54.308177 31613 raft_consensus.cc:374] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:54.308354 31613 raft_consensus.cc:729] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33ae524031954fc69f9d737eeb2a9408, State: Initialized, Role: FOLLOWER
I20260810 14:44:54.309218 31613 consensus_queue.cc:260] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4120b838370e43dabfe66376b80b1dc6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 40327 } } peers { permanent_uuid: "9095ed6a918c4ee483876b42b95f6444" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36431 } } peers { permanent_uuid: "33ae524031954fc69f9d737eeb2a9408" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 43921 } }
I20260810 14:44:54.310477 31593 heartbeater.cc:503] Master 127.24.215.190:46289 was elected leader, sending a full tablet report...
I20260810 14:44:54.311269 31613 ts_tablet_manager.cc:1428] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408: Time spent starting tablet: real 0.015s user 0.014s sys 0.001s
I20260810 14:44:54.311259 31612 ts_tablet_manager.cc:1428] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6: Time spent starting tablet: real 0.023s user 0.006s sys 0.014s
I20260810 14:44:54.311138 31616 raft_consensus.cc:348] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4120b838370e43dabfe66376b80b1dc6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 40327 } } peers { permanent_uuid: "9095ed6a918c4ee483876b42b95f6444" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36431 } } peers { permanent_uuid: "33ae524031954fc69f9d737eeb2a9408" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 43921 } }
I20260810 14:44:54.311578 31616 raft_consensus.cc:374] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:54.311682 31616 raft_consensus.cc:729] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9095ed6a918c4ee483876b42b95f6444, State: Initialized, Role: FOLLOWER
I20260810 14:44:54.312378 31616 consensus_queue.cc:260] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4120b838370e43dabfe66376b80b1dc6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 40327 } } peers { permanent_uuid: "9095ed6a918c4ee483876b42b95f6444" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36431 } } peers { permanent_uuid: "33ae524031954fc69f9d737eeb2a9408" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 43921 } }
I20260810 14:44:54.313606 31435 heartbeater.cc:503] Master 127.24.215.190:46289 was elected leader, sending a full tablet report...
I20260810 14:44:54.314190 31616 ts_tablet_manager.cc:1428] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444: Time spent starting tablet: real 0.013s user 0.008s sys 0.005s
W20260810 14:44:54.443037 31594 tablet.cc:2369] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:44:54.489002 31436 tablet.cc:2369] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:44:54.491994 31621 raft_consensus.cc:482] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:44:54.492306 31621 raft_consensus.cc:504] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4120b838370e43dabfe66376b80b1dc6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 40327 } } peers { permanent_uuid: "9095ed6a918c4ee483876b42b95f6444" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36431 } } peers { permanent_uuid: "33ae524031954fc69f9d737eeb2a9408" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 43921 } }
I20260810 14:44:54.494102 31621 leader_election.cc:288] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4120b838370e43dabfe66376b80b1dc6 (127.24.215.129:40327), 9095ed6a918c4ee483876b42b95f6444 (127.24.215.130:36431)
I20260810 14:44:54.496940 31595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.494307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:40327 (local address 127.24.215.131:54267)
0810 14:44:54.494551 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:54.494573 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:54.494689 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:54.495294 (+ 605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:54.495305 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:54.495329 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:54.496087 (+ 758us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:44:54.496090 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:54.496108 (+ 18us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:44:54.496176 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:54.496514 (+ 338us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:54.496518 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:54.496523 (+ 5us) client_negotiation.cc:769] Sending connection context
0810 14:44:54.496627 (+ 104us) client_negotiation.cc:240] Negotiation successful
0810 14:44:54.496682 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260810 14:44:54.496994 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.494609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:54267 (local address 127.24.215.129:40327)
0810 14:44:54.494802 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:54.494808 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:54.494827 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:44:54.494895 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:54.494902 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:54.494989 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:54.495123 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:54.496343 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:54.496348 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:54.496351 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:54.496367 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:54.496403 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:54.496415 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:54.496540 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:54.496625 (+ 85us) server_negotiation.cc:299] Negotiation successful
0810 14:44:54.496687 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 14:44:54.497395 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.494995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:40677 (local address 127.24.215.130:36431)
0810 14:44:54.495202 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:54.495208 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:54.495449 (+ 241us) server_negotiation.cc:407] Connection header received
0810 14:44:54.495561 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:54.495569 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:54.495635 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:54.495725 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:54.496705 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:54.496711 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:54.496715 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:54.496732 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:54.496769 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:54.496781 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:54.496901 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:54.497052 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 14:44:54.497113 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:44:54.497565 31623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.494699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:36431 (local address 127.24.215.131:40677)
0810 14:44:54.495339 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:54.495359 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:54.495455 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:54.495777 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:54.495783 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:54.495809 (+ 26us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:54.496510 (+ 701us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:44:54.496512 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:54.496525 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:44:54.496561 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:54.496918 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:54.496929 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:54.496937 (+ 8us) client_negotiation.cc:769] Sending connection context
0810 14:44:54.497062 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 14:44:54.497110 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":293,"threads_started":1}
I20260810 14:44:54.498294 31618 raft_consensus.cc:482] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:44:54.498646 31618 raft_consensus.cc:504] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4120b838370e43dabfe66376b80b1dc6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 40327 } } peers { permanent_uuid: "9095ed6a918c4ee483876b42b95f6444" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36431 } } peers { permanent_uuid: "33ae524031954fc69f9d737eeb2a9408" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 43921 } }
I20260810 14:44:54.498674 31382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64a89918c8e419da4be9c2c680946c9" candidate_uuid: "33ae524031954fc69f9d737eeb2a9408" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9095ed6a918c4ee483876b42b95f6444" is_pre_election: true
I20260810 14:44:54.498675 31218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64a89918c8e419da4be9c2c680946c9" candidate_uuid: "33ae524031954fc69f9d737eeb2a9408" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4120b838370e43dabfe66376b80b1dc6" is_pre_election: true
I20260810 14:44:54.499281 31382 raft_consensus.cc:2454] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33ae524031954fc69f9d737eeb2a9408 in term 0.
I20260810 14:44:54.499461 31218 raft_consensus.cc:2454] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 33ae524031954fc69f9d737eeb2a9408 in term 0.
I20260810 14:44:54.500012 31461 leader_election.cc:302] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33ae524031954fc69f9d737eeb2a9408, 9095ed6a918c4ee483876b42b95f6444; no voters:
I20260810 14:44:54.500417 31621 raft_consensus.cc:2781] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:44:54.500587 31621 raft_consensus.cc:482] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:44:54.500638 31618 leader_election.cc:288] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9095ed6a918c4ee483876b42b95f6444 (127.24.215.130:36431), 33ae524031954fc69f9d737eeb2a9408 (127.24.215.131:43921)
I20260810 14:44:54.500715 31621 raft_consensus.cc:3037] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:54.504151 31621 raft_consensus.cc:504] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4120b838370e43dabfe66376b80b1dc6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 40327 } } peers { permanent_uuid: "9095ed6a918c4ee483876b42b95f6444" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36431 } } peers { permanent_uuid: "33ae524031954fc69f9d737eeb2a9408" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 43921 } }
I20260810 14:44:54.504421 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.501085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:47991 (local address 127.24.215.130:36431)
0810 14:44:54.502125 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:54.502130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:54.502146 (+ 16us) server_negotiation.cc:407] Connection header received
0810 14:44:54.502191 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:54.502195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:54.502255 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:54.502353 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:54.503787 (+ 1434us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:54.503792 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:54.503797 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:54.503813 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:54.503850 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:54.503860 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:54.503985 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:54.504063 (+ 78us) server_negotiation.cc:299] Negotiation successful
0810 14:44:54.504121 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":929}
I20260810 14:44:54.504483 31609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.501706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:44099 (local address 127.24.215.131:43921)
0810 14:44:54.501859 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:54.501863 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:54.502017 (+ 154us) server_negotiation.cc:407] Connection header received
0810 14:44:54.502230 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:54.502235 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:54.502291 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:54.502352 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:54.503752 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:54.503758 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:54.503762 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:54.503782 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:54.503817 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:54.503833 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:54.503940 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:54.504199 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 14:44:54.504255 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 14:44:54.504571 31624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.500777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:36431 (local address 127.24.215.129:47991)
0810 14:44:54.501506 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:54.501531 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:54.501643 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:54.502634 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:54.502638 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:54.502654 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:54.503603 (+ 949us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:44:54.503605 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:54.503618 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:44:54.503652 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:54.503958 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:54.503963 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:54.503970 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 14:44:54.504108 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 14:44:54.504158 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":378,"threads_started":1}
I20260810 14:44:54.505151 31625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.501263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:43921 (local address 127.24.215.129:44099)
0810 14:44:54.501919 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:54.501937 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:54.502092 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:54.502573 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:54.502577 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:54.502601 (+ 24us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:44:54.503525 (+ 924us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:44:54.503529 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:44:54.503550 (+ 21us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:44:54.503596 (+ 46us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:44:54.504042 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:44:54.504050 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:44:54.504054 (+ 4us) client_negotiation.cc:769] Sending connection context
0810 14:44:54.504687 (+ 633us) client_negotiation.cc:240] Negotiation successful
0810 14:44:54.504738 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":1141,"threads_started":1}
I20260810 14:44:54.505125 31621 leader_election.cc:288] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [CANDIDATE]: Term 1 election: Requested vote from peers 4120b838370e43dabfe66376b80b1dc6 (127.24.215.129:40327), 9095ed6a918c4ee483876b42b95f6444 (127.24.215.130:36431)
I20260810 14:44:54.505157 31382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64a89918c8e419da4be9c2c680946c9" candidate_uuid: "4120b838370e43dabfe66376b80b1dc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9095ed6a918c4ee483876b42b95f6444" is_pre_election: true
I20260810 14:44:54.505381 31382 raft_consensus.cc:2454] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4120b838370e43dabfe66376b80b1dc6 in term 0.
I20260810 14:44:54.505594 31218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64a89918c8e419da4be9c2c680946c9" candidate_uuid: "33ae524031954fc69f9d737eeb2a9408" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4120b838370e43dabfe66376b80b1dc6"
I20260810 14:44:54.505735 31218 raft_consensus.cc:3037] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:54.506131 31134 leader_election.cc:302] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4120b838370e43dabfe66376b80b1dc6, 9095ed6a918c4ee483876b42b95f6444; no voters:
I20260810 14:44:54.506325 31539 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64a89918c8e419da4be9c2c680946c9" candidate_uuid: "4120b838370e43dabfe66376b80b1dc6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "33ae524031954fc69f9d737eeb2a9408" is_pre_election: true
I20260810 14:44:54.506589 31382 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b64a89918c8e419da4be9c2c680946c9" candidate_uuid: "33ae524031954fc69f9d737eeb2a9408" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9095ed6a918c4ee483876b42b95f6444"
I20260810 14:44:54.506738 31382 raft_consensus.cc:3037] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:54.506798 31539 raft_consensus.cc:2379] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4120b838370e43dabfe66376b80b1dc6 in current term 1: Already voted for candidate 33ae524031954fc69f9d737eeb2a9408 in this term.
I20260810 14:44:54.508682 31218 raft_consensus.cc:2454] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33ae524031954fc69f9d737eeb2a9408 in term 1.
I20260810 14:44:54.508868 31618 raft_consensus.cc:2741] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260810 14:44:54.509042 31461 leader_election.cc:302] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 33ae524031954fc69f9d737eeb2a9408, 4120b838370e43dabfe66376b80b1dc6; no voters:
I20260810 14:44:54.509076 31382 raft_consensus.cc:2454] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 33ae524031954fc69f9d737eeb2a9408 in term 1.
I20260810 14:44:54.509294 31621 raft_consensus.cc:2781] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:54.509793 31621 raft_consensus.cc:686] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [term 1 LEADER]: Becoming Leader. State: Replica: 33ae524031954fc69f9d737eeb2a9408, State: Running, Role: LEADER
I20260810 14:44:54.510275 31621 consensus_queue.cc:237] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4120b838370e43dabfe66376b80b1dc6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 40327 } } peers { permanent_uuid: "9095ed6a918c4ee483876b42b95f6444" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36431 } } peers { permanent_uuid: "33ae524031954fc69f9d737eeb2a9408" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 43921 } }
I20260810 14:44:54.515166 31044 catalog_manager.cc:5495] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 reported cstate change: term changed from 0 to 1, leader changed from <none> to 33ae524031954fc69f9d737eeb2a9408 (127.24.215.131). New cstate: current_term: 1 leader_uuid: "33ae524031954fc69f9d737eeb2a9408" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4120b838370e43dabfe66376b80b1dc6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 40327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9095ed6a918c4ee483876b42b95f6444" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36431 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "33ae524031954fc69f9d737eeb2a9408" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 43921 } health_report { overall_health: HEALTHY } } }
W20260810 14:44:54.535877 31270 tablet.cc:2369] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:44:54.549149 31609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.547173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48296 (local address 127.24.215.131:43921)
0810 14:44:54.547340 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:54.547345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:54.547458 (+ 113us) server_negotiation.cc:407] Connection header received
0810 14:44:54.547633 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:54.547636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:54.547733 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:54.547820 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:54.548692 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:54.548696 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:54.548699 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:54.548711 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:54.548737 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:54.548748 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:54.548845 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:54.548888 (+ 43us) server_negotiation.cc:299] Negotiation successful
0810 14:44:54.548934 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 14:44:54.557410 31218 raft_consensus.cc:1261] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6 [term 1 FOLLOWER]: Refusing update from remote peer 33ae524031954fc69f9d737eeb2a9408: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:54.557415 31382 raft_consensus.cc:1261] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444 [term 1 FOLLOWER]: Refusing update from remote peer 33ae524031954fc69f9d737eeb2a9408: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:44:54.558279 31626 consensus_queue.cc:1035] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4120b838370e43dabfe66376b80b1dc6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 40327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:54.558687 31621 consensus_queue.cc:1035] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9095ed6a918c4ee483876b42b95f6444" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 36431 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:44:54.591149 31042 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:44:54.591483 31042 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48984:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:44:54.595115 31042 catalog_manager.cc:5782] T 00000000000000000000000000000000 P eb9cf27daec24089bb31bedfd5409adb: Sending DeleteTablet for 3 replicas of tablet b64a89918c8e419da4be9c2c680946c9
I20260810 14:44:54.596386 31352 tablet_service.cc:1514] Processing DeleteTablet for tablet b64a89918c8e419da4be9c2c680946c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:54 UTC) from {username='slave'} at 127.0.0.1:42460
I20260810 14:44:54.596518 31513 tablet_service.cc:1514] Processing DeleteTablet for tablet b64a89918c8e419da4be9c2c680946c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:54 UTC) from {username='slave'} at 127.0.0.1:48280
I20260810 14:44:54.597441 31190 tablet_service.cc:1514] Processing DeleteTablet for tablet b64a89918c8e419da4be9c2c680946c9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:44:54 UTC) from {username='slave'} at 127.0.0.1:60850
I20260810 14:44:54.603611 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 31110
I20260810 14:44:54.611937 31642 tablet_replica.cc:330] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408: stopping tablet replica
I20260810 14:44:54.612985 31642 raft_consensus.cc:2229] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:44:54.613829 31642 raft_consensus.cc:2258] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:54.616039 31642 ts_tablet_manager.cc:1905] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:54.623732 31642 ts_tablet_manager.cc:1918] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:44:54.623981 31642 log.cc:1196] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/b64a89918c8e419da4be9c2c680946c9
I20260810 14:44:54.624611 31642 ts_tablet_manager.cc:1939] T b64a89918c8e419da4be9c2c680946c9 P 33ae524031954fc69f9d737eeb2a9408: Deleting consensus metadata
I20260810 14:44:54.626789 31026 catalog_manager.cc:4841] TS 33ae524031954fc69f9d737eeb2a9408 (127.24.215.131:43921): tablet b64a89918c8e419da4be9c2c680946c9 (table test-table [id=861eaa536fa047fea9d85395a840461f]) successfully deleted
I20260810 14:44:54.628594 31643 tablet_replica.cc:330] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6: stopping tablet replica
I20260810 14:44:54.629148 31643 raft_consensus.cc:2229] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:44:54.630303 31607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.605467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60854 (local address 127.24.215.129:40327)
0810 14:44:54.605709 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:54.605719 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:54.627510 (+ 21791us) server_negotiation.cc:407] Connection header received
0810 14:44:54.627762 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:54.627769 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:54.627893 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:54.628030 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:54.629289 (+ 1259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:54.629297 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:54.629302 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:54.629324 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:54.629364 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:54.629381 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:54.629493 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:54.629741 (+ 248us) server_negotiation.cc:299] Negotiation successful
0810 14:44:54.629807 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 14:44:54.629945 31643 raft_consensus.cc:2258] T b64a89918c8e419da4be9c2c680946c9 P 4120b838370e43dabfe66376b80b1dc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:54.631402 31264 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:54.632171 31640 tablet_replica.cc:330] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444: stopping tablet replica
I20260810 14:44:54.632550 31640 raft_consensus.cc:2229] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:44:54.632977 31640 raft_consensus.cc:2258] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:44:54.635195 31640 ts_tablet_manager.cc:1905] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:44:54.643847 31640 ts_tablet_manager.cc:1918] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:44:54.644074 31640 log.cc:1196] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/b64a89918c8e419da4be9c2c680946c9
I20260810 14:44:54.644646 31640 ts_tablet_manager.cc:1939] T b64a89918c8e419da4be9c2c680946c9 P 9095ed6a918c4ee483876b42b95f6444: Deleting consensus metadata
I20260810 14:44:54.646827 31027 catalog_manager.cc:4841] TS 9095ed6a918c4ee483876b42b95f6444 (127.24.215.130:36431): tablet b64a89918c8e419da4be9c2c680946c9 (table test-table [id=861eaa536fa047fea9d85395a840461f]) successfully deleted
I20260810 14:44:54.727541 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 31110
W20260810 14:44:54.751353 31027 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.215.129:40327 (error 108)
I20260810 14:44:54.751531 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 31274
I20260810 14:44:54.753288 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.752822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:40327 (local address 127.0.0.1:60868)
0810 14:44:54.752964 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:54.753114 (+ 150us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:40327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260810 14:44:54.753506 31027 catalog_manager.cc:4584] TS 4120b838370e43dabfe66376b80b1dc6 (127.24.215.129:40327): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b64a89918c8e419da4be9c2c680946c9: Network error: Client connection negotiation failed: client connection to 127.24.215.129:40327: connect: Connection refused (error 111)
I20260810 14:44:54.755272 31611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.752578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42468 (local address 127.24.215.130:36431)
0810 14:44:54.752790 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:54.752797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:54.752910 (+ 113us) server_negotiation.cc:407] Connection header received
0810 14:44:54.753186 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:54.753193 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:54.753280 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:54.753354 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:54.754510 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:54.754517 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:54.754522 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:54.754541 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:54.754581 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:54.754596 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:54.754721 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:54.754924 (+ 203us) server_negotiation.cc:299] Negotiation successful
0810 14:44:54.754982 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 14:44:54.756426 31429 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:54.807324 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.806719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:40327 (local address 127.0.0.1:60872)
0810 14:44:54.806986 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:54.807127 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:40327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
I20260810 14:44:54.808573 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.808149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:40327 (local address 127.0.0.1:60884)
0810 14:44:54.808320 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:54.808437 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:40327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 14:44:54.861104 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 31274
I20260810 14:44:54.869472 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.868866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:40327 (local address 127.0.0.1:60896)
0810 14:44:54.869141 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:54.869271 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:40327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 14:44:54.870697 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.870262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:40327 (local address 127.0.0.1:60898)
0810 14:44:54.870447 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:54.870537 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:40327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 14:44:54.882514 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 31439
I20260810 14:44:54.885654 31609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.883380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48310 (local address 127.24.215.131:43921)
0810 14:44:54.883596 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:54.883601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:54.883621 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:44:54.883815 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:54.883821 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:54.883891 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:54.884011 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:54.884984 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:54.884991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:54.884996 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:54.885015 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:54.885054 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:54.885065 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:54.885212 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:54.885294 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 14:44:54.885358 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 14:44:54.886454 31588 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:54.952013 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.951512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:40327 (local address 127.0.0.1:60904)
0810 14:44:54.951745 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:54.951841 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:40327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 14:44:54.953138 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:54.952701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:40327 (local address 127.0.0.1:60908)
0810 14:44:54.952922 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:54.953002 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:40327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260810 14:44:55.007195 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 31439
I20260810 14:44:55.029753 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 31005
I20260810 14:44:55.030933 31078 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:55.137595 31608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:55.136975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:40327 (local address 127.0.0.1:60912)
0810 14:44:55.137220 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:55.137362 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:40327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260810 14:44:55.137985 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 31005
2026-08-10T14:44:55Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2890 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260810 14:44:55.185101 25438 mini_oidc.cc:152] Starting JWKS server
I20260810 14:44:55.189616 25438 webserver.cc:460] Webserver started at https://127.0.0.1:39559/ using document root <none> and password file <none>
I20260810 14:44:55.190027 25438 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 14:44:55.190783 25438 webserver.cc:460] Webserver started at http://127.0.0.1:46009/ using document root <none> and password file <none>
2026-08-10T14:44:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:44:55Z Disabled control of system clock
I20260810 14:44:55.221625 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:45619
--webserver_interface=127.24.215.190
--webserver_port=0
--builtin_ntp_servers=127.24.215.148:38243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:45619
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39559/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/testchainca.cert with env {}
W20260810 14:44:55.562986 31655 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:55.563383 31655 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:55.563480 31655 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:55.570905 31655 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:55.571017 31655 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:55.571055 31655 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:55.571177 31655 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:55.571225 31655 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 14:44:55.571282 31655 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 14:44:55.571331 31655 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:39559/jwks.json
I20260810 14:44:55.581651 31655 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:38243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:45619
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:45619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:39559/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:55.582983 31655 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:55.584347 31655 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:55.599017 31661 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:55.598963 31662 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:55.599224 31664 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:55.599906 31655 server_base.cc:1034] running on GCE node
I20260810 14:44:55.600755 31655 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:55.602269 31655 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:55.603498 31655 hybrid_clock.cc:648] HybridClock initialized: now 1786373095603452 us; error 36 us; skew 500 ppm
I20260810 14:44:55.603919 31655 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:44:55.606273 31655 webserver.cc:460] Webserver started at http://127.24.215.190:44443/ using document root <none> and password file <none>
I20260810 14:44:55.607007 31655 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:55.607162 31655 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:55.607501 31655 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:55.610458 31655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "52fbf95ea28a4110b3cbc19c1e5b856e"
format_stamp: "Formatted at 2026-08-10 14:44:55 on dist-test-slave-vkjp"
server_key: "a4890e88a80ce6120ae42b541a81f446"
server_key_iv: "b1c15e68805db0aec164ad58339887da"
server_key_version: "encryptionkey@0"
I20260810 14:44:55.611423 31655 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "52fbf95ea28a4110b3cbc19c1e5b856e"
format_stamp: "Formatted at 2026-08-10 14:44:55 on dist-test-slave-vkjp"
server_key: "a4890e88a80ce6120ae42b541a81f446"
server_key_iv: "b1c15e68805db0aec164ad58339887da"
server_key_version: "encryptionkey@0"
I20260810 14:44:55.616703 31655 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:44:55.620246 31673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:55.621676 31655 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 14:44:55.621877 31655 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "52fbf95ea28a4110b3cbc19c1e5b856e"
format_stamp: "Formatted at 2026-08-10 14:44:55 on dist-test-slave-vkjp"
server_key: "a4890e88a80ce6120ae42b541a81f446"
server_key_iv: "b1c15e68805db0aec164ad58339887da"
server_key_version: "encryptionkey@0"
I20260810 14:44:55.622115 31655 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:55.653822 31655 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:55.654754 31655 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:55.655112 31655 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:55.674172 31655 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:45619
I20260810 14:44:55.674191 31738 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:45619 every 8 connection(s)
I20260810 14:44:55.675575 31655 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:44:55.679373 31739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:55.682753 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 31655
I20260810 14:44:55.682993 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:44:55.683650 25438 external_mini_cluster.cc:1428] Setting key 8ea324a28226cc3820ce017e30abde6c
I20260810 14:44:55.693071 31739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e: Bootstrap starting.
I20260810 14:44:55.696981 31739 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:55.698324 31739 log.cc:824] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:55.698938 31743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:55.685585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48350 (local address 127.24.215.190:45619)
0810 14:44:55.686676 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:55.686691 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:55.686742 (+ 51us) server_negotiation.cc:407] Connection header received
0810 14:44:55.688478 (+ 1736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:55.688512 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:55.689126 (+ 614us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:55.689580 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:55.690133 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:55.692178 (+ 2045us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:55.693607 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:55.694611 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:55.694839 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:55.696010 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:55.696021 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:55.696040 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:55.696742 (+ 702us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:55.696873 (+ 131us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:55.696902 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:55.697032 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:55.697256 (+ 224us) server_negotiation.cc:299] Negotiation successful
0810 14:44:55.697632 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":353,"threads_started":1}
I20260810 14:44:55.701351 31739 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e: No bootstrap required, opened a new log
I20260810 14:44:55.702844 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:44:55.709008 31739 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52fbf95ea28a4110b3cbc19c1e5b856e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45619 } }
I20260810 14:44:55.709345 31739 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:55.709476 31739 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52fbf95ea28a4110b3cbc19c1e5b856e, State: Initialized, Role: FOLLOWER
I20260810 14:44:55.710085 31739 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52fbf95ea28a4110b3cbc19c1e5b856e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45619 } }
I20260810 14:44:55.710332 31739 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:44:55.710469 31739 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:44:55.710646 31739 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:55.712918 31739 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52fbf95ea28a4110b3cbc19c1e5b856e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45619 } }
I20260810 14:44:55.713428 31739 leader_election.cc:302] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52fbf95ea28a4110b3cbc19c1e5b856e; no voters:
I20260810 14:44:55.714010 31739 leader_election.cc:288] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:44:55.714145 31746 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:55.714943 31746 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [term 1 LEADER]: Becoming Leader. State: Replica: 52fbf95ea28a4110b3cbc19c1e5b856e, State: Running, Role: LEADER
I20260810 14:44:55.715539 31746 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52fbf95ea28a4110b3cbc19c1e5b856e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45619 } }
I20260810 14:44:55.716167 31739 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:44:55.718214 31747 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52fbf95ea28a4110b3cbc19c1e5b856e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52fbf95ea28a4110b3cbc19c1e5b856e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45619 } } }
I20260810 14:44:55.718695 31747 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:55.718349 31748 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52fbf95ea28a4110b3cbc19c1e5b856e. Latest consensus state: current_term: 1 leader_uuid: "52fbf95ea28a4110b3cbc19c1e5b856e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52fbf95ea28a4110b3cbc19c1e5b856e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45619 } } }
I20260810 14:44:55.719173 31748 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:55.720696 31753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:44:55.725690 31753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:44:55.735018 31753 catalog_manager.cc:1347] Generated new cluster ID: 32023f671e294741992cced4fb920b93
I20260810 14:44:55.735172 31753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:44:55.749414 31753 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:44:55.751255 31753 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:44:55.762288 31753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52fbf95ea28a4110b3cbc19c1e5b856e: Generated new TSK 0
I20260810 14:44:55.763208 31753 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 14:44:55.808087 31775 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260810 14:44:55.808261 31775 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 14:44:55.810776 31743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:55.785270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48356 (local address 127.24.215.190:45619)
0810 14:44:55.785483 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:55.785489 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:55.785923 (+ 434us) server_negotiation.cc:407] Connection header received
0810 14:44:55.786291 (+ 368us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:55.786297 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:55.786404 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:55.786566 (+ 162us) server_negotiation.cc:226] Negotiated authn=JWT
0810 14:44:55.787106 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:55.788267 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:55.789690 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:55.790427 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:55.790553 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:55.791142 (+ 589us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 14:44:55.810073 (+ 18931us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 14:44:55.810457 (+ 384us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:48356: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":84}
W20260810 14:44:55.810900 31743 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:48356: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260810 14:44:55.812405 31774 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:55.785141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:45619 (local address 127.0.0.1:48356)
0810 14:44:55.785811 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:55.785828 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:55.785989 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:55.786620 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:55.786624 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:55.786635 (+ 11us) client_negotiation.cc:189] Negotiated authn=JWT
0810 14:44:55.786941 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:55.786949 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:55.788409 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:55.788413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:55.789546 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:55.789557 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:55.790623 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:55.790628 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:55.790758 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:55.790816 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 14:44:55.810378 (+ 19562us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0810 14:44:55.810656 (+ 278us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.215.190:45619: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":505,"thread_start_us":343,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.24.215.190:45619: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:39559/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260810 14:44:55.814240 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 31655
I20260810 14:44:55.815439 31732 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:55.908692 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 31655
2026-08-10T14:44:55Z chronyd exiting
I20260810 14:44:55.962778 25438 test_util.cc:175] -----------------------------------------------
I20260810 14:44:55.962965 25438 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786373051600012-25438-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (784 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260810 14:44:55.966722 25438 mini_oidc.cc:152] Starting JWKS server
I20260810 14:44:55.968660 25438 webserver.cc:460] Webserver started at https://127.0.0.1:33291/ using document root <none> and password file <none>
I20260810 14:44:55.968917 25438 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 14:44:55.969599 25438 webserver.cc:460] Webserver started at http://127.0.0.1:42091/ using document root <none> and password file <none>
2026-08-10T14:44:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:44:55Z Disabled control of system clock
I20260810 14:44:55.998857 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:35251
--webserver_interface=127.24.215.190
--webserver_port=0
--builtin_ntp_servers=127.24.215.148:43875
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:35251
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33291/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/testchainca.cert with env {}
W20260810 14:44:56.326114 31783 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:56.326478 31783 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:56.326648 31783 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:56.334694 31783 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:56.334832 31783 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:56.334906 31783 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:56.334991 31783 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:56.335080 31783 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 14:44:56.335186 31783 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 14:44:56.335246 31783 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33291/jwks.json
I20260810 14:44:56.345119 31783 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:43875
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:35251
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:35251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33291/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:56.346488 31783 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:56.347846 31783 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:56.360229 31788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:56.360163 31789 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:56.360158 31792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:56.360975 31783 server_base.cc:1034] running on GCE node
I20260810 14:44:56.361788 31783 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:56.363229 31783 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:56.364467 31783 hybrid_clock.cc:648] HybridClock initialized: now 1786373096364375 us; error 82 us; skew 500 ppm
I20260810 14:44:56.364811 31783 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:44:56.372789 31783 webserver.cc:460] Webserver started at http://127.24.215.190:37913/ using document root <none> and password file <none>
I20260810 14:44:56.373543 31783 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:56.373687 31783 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:56.374037 31783 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:56.377854 31783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "0dcc2ec6321a4af4bfdd72fd6db48ac5"
format_stamp: "Formatted at 2026-08-10 14:44:56 on dist-test-slave-vkjp"
server_key: "4f48d5ef0cb3e3862110c10e042eb55c"
server_key_iv: "5ac47c55cf0a838baf6597eb8b26bc46"
server_key_version: "encryptionkey@0"
I20260810 14:44:56.378907 31783 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "0dcc2ec6321a4af4bfdd72fd6db48ac5"
format_stamp: "Formatted at 2026-08-10 14:44:56 on dist-test-slave-vkjp"
server_key: "4f48d5ef0cb3e3862110c10e042eb55c"
server_key_iv: "5ac47c55cf0a838baf6597eb8b26bc46"
server_key_version: "encryptionkey@0"
I20260810 14:44:56.384109 31783 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260810 14:44:56.387512 31798 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:56.388851 31783 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 14:44:56.389037 31783 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "0dcc2ec6321a4af4bfdd72fd6db48ac5"
format_stamp: "Formatted at 2026-08-10 14:44:56 on dist-test-slave-vkjp"
server_key: "4f48d5ef0cb3e3862110c10e042eb55c"
server_key_iv: "5ac47c55cf0a838baf6597eb8b26bc46"
server_key_version: "encryptionkey@0"
I20260810 14:44:56.389245 31783 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:56.408185 31783 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:56.408968 31783 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:56.409281 31783 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:56.427745 31783 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:35251
I20260810 14:44:56.427781 31862 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:35251 every 8 connection(s)
I20260810 14:44:56.429183 31783 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:44:56.432843 31863 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:56.438858 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 31783
I20260810 14:44:56.439050 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:44:56.439751 25438 external_mini_cluster.cc:1428] Setting key 6562ffc52699c9ac0b3aeb242e049f76
I20260810 14:44:56.443588 31863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5: Bootstrap starting.
I20260810 14:44:56.446880 31863 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:56.448200 31863 log.cc:824] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:56.451812 31863 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5: No bootstrap required, opened a new log
I20260810 14:44:56.454075 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:56.441803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40714 (local address 127.24.215.190:35251)
0810 14:44:56.442763 (+ 960us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:56.442777 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:56.442821 (+ 44us) server_negotiation.cc:407] Connection header received
0810 14:44:56.444564 (+ 1743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:56.444588 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:56.445038 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:56.445356 (+ 318us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:56.445954 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:56.447382 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:56.449163 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:56.450411 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:56.450593 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:56.451784 (+ 1191us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:56.451791 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:56.451811 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:56.452404 (+ 593us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:56.452478 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:56.452504 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:56.452598 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:56.452827 (+ 229us) server_negotiation.cc:299] Negotiation successful
0810 14:44:56.453147 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":357,"threads_started":1}
I20260810 14:44:56.457934 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:44:56.460259 31863 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dcc2ec6321a4af4bfdd72fd6db48ac5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35251 } }
I20260810 14:44:56.460644 31863 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:56.460760 31863 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dcc2ec6321a4af4bfdd72fd6db48ac5, State: Initialized, Role: FOLLOWER
I20260810 14:44:56.461369 31863 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dcc2ec6321a4af4bfdd72fd6db48ac5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35251 } }
I20260810 14:44:56.461616 31863 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:44:56.461756 31863 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:44:56.461946 31863 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:56.464339 31863 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dcc2ec6321a4af4bfdd72fd6db48ac5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35251 } }
I20260810 14:44:56.464862 31863 leader_election.cc:302] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0dcc2ec6321a4af4bfdd72fd6db48ac5; no voters:
I20260810 14:44:56.465447 31863 leader_election.cc:288] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:44:56.465617 31869 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:56.466343 31869 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [term 1 LEADER]: Becoming Leader. State: Replica: 0dcc2ec6321a4af4bfdd72fd6db48ac5, State: Running, Role: LEADER
I20260810 14:44:56.466907 31869 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dcc2ec6321a4af4bfdd72fd6db48ac5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35251 } }
I20260810 14:44:56.467558 31863 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:44:56.469699 31871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0dcc2ec6321a4af4bfdd72fd6db48ac5. Latest consensus state: current_term: 1 leader_uuid: "0dcc2ec6321a4af4bfdd72fd6db48ac5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dcc2ec6321a4af4bfdd72fd6db48ac5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35251 } } }
I20260810 14:44:56.470139 31871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:56.471654 31870 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0dcc2ec6321a4af4bfdd72fd6db48ac5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0dcc2ec6321a4af4bfdd72fd6db48ac5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35251 } } }
I20260810 14:44:56.471982 31870 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5 [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:56.472496 31878 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:44:56.476940 31878 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:44:56.485416 31878 catalog_manager.cc:1347] Generated new cluster ID: 6435e4cdd1e7451ea03c44dde844cbab
I20260810 14:44:56.485540 31878 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:44:56.508204 31878 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:44:56.509601 31878 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:44:56.525671 31878 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0dcc2ec6321a4af4bfdd72fd6db48ac5: Generated new TSK 0
I20260810 14:44:56.526546 31878 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260810 14:44:56.572264 31896 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260810 14:44:56.572407 31896 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 14:44:56.573848 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:56.545539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40726 (local address 127.24.215.190:35251)
0810 14:44:56.545790 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:56.545797 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:56.546177 (+ 380us) server_negotiation.cc:407] Connection header received
0810 14:44:56.546451 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:56.546455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:56.546523 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:56.546655 (+ 132us) server_negotiation.cc:226] Negotiated authn=JWT
0810 14:44:56.547280 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:56.548009 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:56.550060 (+ 2051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:56.550545 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:56.550641 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:56.551289 (+ 648us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 14:44:56.573071 (+ 21782us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33291/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 14:44:56.573529 (+ 458us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40726: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33291/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":74}
W20260810 14:44:56.573988 31867 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40726: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33291/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260810 14:44:56.574074 31895 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:56.545413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35251 (local address 127.0.0.1:40726)
0810 14:44:56.546056 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:56.546076 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:56.546239 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:56.546706 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:56.546712 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:56.546725 (+ 13us) client_negotiation.cc:189] Negotiated authn=JWT
0810 14:44:56.547073 (+ 348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:56.547111 (+ 38us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:56.548145 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:56.548151 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:56.549899 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:56.549915 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:56.550709 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:56.550716 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:56.550924 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:56.550983 (+ 59us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 14:44:56.573347 (+ 22364us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33291/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0810 14:44:56.573624 (+ 277us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.215.190:35251: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33291/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":338,"threads_started":1}
I20260810 14:44:56.575490 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 31783
I20260810 14:44:56.576756 31854 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:56.672067 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 31783
2026-08-10T14:44:56Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (763 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260810 14:44:56.730702 25438 mini_oidc.cc:152] Starting JWKS server
I20260810 14:44:56.732636 25438 webserver.cc:460] Webserver started at https://127.0.0.1:42219/ using document root <none> and password file <none>
I20260810 14:44:56.732841 25438 mini_oidc.cc:189] Starting OIDC Discovery server
I20260810 14:44:56.733403 25438 webserver.cc:460] Webserver started at http://127.0.0.1:39119/ using document root <none> and password file <none>
2026-08-10T14:44:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:44:56Z Disabled control of system clock
I20260810 14:44:56.762838 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:39619
--webserver_interface=127.24.215.190
--webserver_port=0
--builtin_ntp_servers=127.24.215.148:39249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:39619
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42219/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 14:44:57.089097 31903 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:57.089464 31903 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:57.089618 31903 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:57.096978 31903 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:57.097123 31903 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:57.097213 31903 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:57.097337 31903 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:57.097426 31903 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 14:44:57.097512 31903 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 14:44:57.097586 31903 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42219/jwks.json
I20260810 14:44:57.106808 31903 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:39249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:39619
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:39619
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42219/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:57.108186 31903 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:57.109992 31903 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:57.121791 31911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:57.121868 31908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:57.121784 31913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:57.122841 31903 server_base.cc:1034] running on GCE node
I20260810 14:44:57.123698 31903 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:57.125038 31903 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:57.126238 31903 hybrid_clock.cc:648] HybridClock initialized: now 1786373097126195 us; error 41 us; skew 500 ppm
I20260810 14:44:57.126549 31903 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:44:57.128899 31903 webserver.cc:460] Webserver started at http://127.24.215.190:36167/ using document root <none> and password file <none>
I20260810 14:44:57.129542 31903 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:57.129629 31903 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:57.129895 31903 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:57.132992 31903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "c9bf8a7af5f141a7a4973337fe83bb18"
format_stamp: "Formatted at 2026-08-10 14:44:57 on dist-test-slave-vkjp"
server_key: "677ff7325f668793ad91419c406c6282"
server_key_iv: "b0f880e6254497552cc8f5fc35ff1b0a"
server_key_version: "encryptionkey@0"
I20260810 14:44:57.133839 31903 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "c9bf8a7af5f141a7a4973337fe83bb18"
format_stamp: "Formatted at 2026-08-10 14:44:57 on dist-test-slave-vkjp"
server_key: "677ff7325f668793ad91419c406c6282"
server_key_iv: "b0f880e6254497552cc8f5fc35ff1b0a"
server_key_version: "encryptionkey@0"
I20260810 14:44:57.138949 31903 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 14:44:57.142373 31919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:57.143779 31903 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:44:57.143924 31903 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "c9bf8a7af5f141a7a4973337fe83bb18"
format_stamp: "Formatted at 2026-08-10 14:44:57 on dist-test-slave-vkjp"
server_key: "677ff7325f668793ad91419c406c6282"
server_key_iv: "b0f880e6254497552cc8f5fc35ff1b0a"
server_key_version: "encryptionkey@0"
I20260810 14:44:57.144104 31903 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:57.170967 31903 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:57.171826 31903 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:57.172096 31903 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:57.190290 31903 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:39619
I20260810 14:44:57.190316 31978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:39619 every 8 connection(s)
I20260810 14:44:57.191728 31903 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:44:57.195566 31979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:57.201649 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 31903
I20260810 14:44:57.201860 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:44:57.202515 25438 external_mini_cluster.cc:1428] Setting key 4d55dd18754cadb987bb6bb66a4648a8
I20260810 14:44:57.205960 31979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18: Bootstrap starting.
I20260810 14:44:57.209313 31979 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:57.210477 31979 log.cc:824] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:57.213557 31979 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18: No bootstrap required, opened a new log
I20260810 14:44:57.216080 31982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:57.204524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45482 (local address 127.24.215.190:39619)
0810 14:44:57.205494 (+ 970us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:57.205513 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:57.205553 (+ 40us) server_negotiation.cc:407] Connection header received
0810 14:44:57.207303 (+ 1750us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:57.207329 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:57.207781 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:57.208096 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:57.208706 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:57.210078 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:57.211778 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:57.212419 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:57.212551 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:57.213780 (+ 1229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:57.213788 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:57.213802 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:57.214334 (+ 532us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:57.214417 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:57.214437 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:57.214523 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:57.214768 (+ 245us) server_negotiation.cc:299] Negotiation successful
0810 14:44:57.215118 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":355,"threads_started":1}
I20260810 14:44:57.219926 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:44:57.222265 31979 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9bf8a7af5f141a7a4973337fe83bb18" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39619 } }
I20260810 14:44:57.222719 31979 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:57.222852 31979 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9bf8a7af5f141a7a4973337fe83bb18, State: Initialized, Role: FOLLOWER
I20260810 14:44:57.223435 31979 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9bf8a7af5f141a7a4973337fe83bb18" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39619 } }
I20260810 14:44:57.223680 31979 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:44:57.223831 31979 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:44:57.224010 31979 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:57.226115 31979 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9bf8a7af5f141a7a4973337fe83bb18" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39619 } }
I20260810 14:44:57.226711 31979 leader_election.cc:302] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9bf8a7af5f141a7a4973337fe83bb18; no voters:
I20260810 14:44:57.227510 31979 leader_election.cc:288] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:44:57.227626 31984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:57.228334 31984 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [term 1 LEADER]: Becoming Leader. State: Replica: c9bf8a7af5f141a7a4973337fe83bb18, State: Running, Role: LEADER
I20260810 14:44:57.228933 31984 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9bf8a7af5f141a7a4973337fe83bb18" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39619 } }
I20260810 14:44:57.229741 31979 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:44:57.231935 31988 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9bf8a7af5f141a7a4973337fe83bb18. Latest consensus state: current_term: 1 leader_uuid: "c9bf8a7af5f141a7a4973337fe83bb18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9bf8a7af5f141a7a4973337fe83bb18" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39619 } } }
I20260810 14:44:57.232324 31988 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:57.232000 31987 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9bf8a7af5f141a7a4973337fe83bb18" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9bf8a7af5f141a7a4973337fe83bb18" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39619 } } }
I20260810 14:44:57.232460 31987 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18 [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:57.236861 31993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:44:57.243089 31993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:44:57.251921 31993 catalog_manager.cc:1347] Generated new cluster ID: bde546f93e8345b58ccbf89cfd19fa93
I20260810 14:44:57.252074 31993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:44:57.267848 31993 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:44:57.269086 31993 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:44:57.274631 31993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9bf8a7af5f141a7a4973337fe83bb18: Generated new TSK 0
I20260810 14:44:57.275571 31993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:44:57.305312 31982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:57.296787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45494 (local address 127.24.215.190:39619)
0810 14:44:57.296994 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:57.297000 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:57.297384 (+ 384us) server_negotiation.cc:407] Connection header received
0810 14:44:57.297609 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:57.297615 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:57.297717 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:57.297887 (+ 170us) server_negotiation.cc:226] Negotiated authn=JWT
0810 14:44:57.298344 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:57.299577 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:57.300960 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:57.301637 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:57.301778 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:57.302238 (+ 460us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 14:44:57.304663 (+ 2425us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42219/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 14:44:57.305002 (+ 339us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45494: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42219/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 14:44:57.305454 31982 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45494: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42219/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260810 14:44:57.306938 32015 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:57.296644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:39619 (local address 127.0.0.1:45494)
0810 14:44:57.297286 (+ 642us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:57.297302 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:57.297419 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:57.297902 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:57.297905 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:57.297915 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0810 14:44:57.298197 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:57.298205 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:57.299713 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:57.299716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:57.300795 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:57.300807 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:57.301819 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:57.301823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:57.301939 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:57.301970 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 14:44:57.304873 (+ 2903us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42219/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0810 14:44:57.306556 (+ 1683us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.215.190:39619: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42219/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":513,"mutex_wait_us":1393,"thread_start_us":364,"threads_started":1}
I20260810 14:44:57.308745 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 31903
I20260810 14:44:57.309926 31974 generic_service.cc:196] Checking for leaks (request via RPC)
E20260810 14:44:57.317094 32017 webserver.cc:533] Webserver: sslize failed:
E20260810 14:44:57.317184 32017 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260810 14:44:57.402283 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 31903
2026-08-10T14:44:57Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (730 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-10T14:44:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:44:57Z Disabled control of system clock
I20260810 14:44:57.490638 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:36407
--webserver_interface=127.24.215.190
--webserver_port=0
--builtin_ntp_servers=127.24.215.148:36183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:36407
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/testchainca.cert with env {}
W20260810 14:44:57.825650 32023 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:44:57.825953 32023 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:44:57.826040 32023 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:44:57.833441 32023 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:44:57.833547 32023 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:44:57.833585 32023 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:44:57.833643 32023 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:44:57.833680 32023 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260810 14:44:57.833742 32023 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260810 14:44:57.833781 32023 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260810 14:44:57.843845 32023 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:36183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:36407
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:36407
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:44:57.845108 32023 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:44:57.846463 32023 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:44:57.859154 32029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:57.859150 32030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:44:57.859105 32032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:44:57.860152 32023 server_base.cc:1034] running on GCE node
I20260810 14:44:57.861011 32023 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:44:57.862497 32023 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:44:57.863725 32023 hybrid_clock.cc:648] HybridClock initialized: now 1786373097863676 us; error 64 us; skew 500 ppm
I20260810 14:44:57.864106 32023 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:44:57.866492 32023 webserver.cc:460] Webserver started at http://127.24.215.190:46545/ using document root <none> and password file <none>
I20260810 14:44:57.867322 32023 fs_manager.cc:359] Metadata directory not provided
I20260810 14:44:57.867465 32023 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:44:57.867777 32023 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:44:57.870667 32023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "52f1cd31c08c4570a92465630675611d"
format_stamp: "Formatted at 2026-08-10 14:44:57 on dist-test-slave-vkjp"
server_key: "dc3b73e64c7b541a9984d09901f9972d"
server_key_iv: "fa3b2a6885c06e8fabe3b303950bd660"
server_key_version: "encryptionkey@0"
I20260810 14:44:57.871549 32023 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "52f1cd31c08c4570a92465630675611d"
format_stamp: "Formatted at 2026-08-10 14:44:57 on dist-test-slave-vkjp"
server_key: "dc3b73e64c7b541a9984d09901f9972d"
server_key_iv: "fa3b2a6885c06e8fabe3b303950bd660"
server_key_version: "encryptionkey@0"
I20260810 14:44:57.876585 32023 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:44:57.880082 32039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:44:57.881476 32023 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 14:44:57.881670 32023 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "52f1cd31c08c4570a92465630675611d"
format_stamp: "Formatted at 2026-08-10 14:44:57 on dist-test-slave-vkjp"
server_key: "dc3b73e64c7b541a9984d09901f9972d"
server_key_iv: "fa3b2a6885c06e8fabe3b303950bd660"
server_key_version: "encryptionkey@0"
I20260810 14:44:57.881886 32023 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:44:57.899578 32023 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:44:57.900446 32023 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:44:57.900738 32023 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:44:57.919571 32023 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:36407
I20260810 14:44:57.919584 32098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:36407 every 8 connection(s)
I20260810 14:44:57.920992 32023 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:44:57.924783 32099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:44:57.930487 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 32023
I20260810 14:44:57.930716 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:44:57.931463 25438 external_mini_cluster.cc:1428] Setting key f61159cc66517e30b3aefab32bd3bd07
I20260810 14:44:57.936458 32099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d: Bootstrap starting.
I20260810 14:44:57.940490 32099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d: Neither blocks nor log segments found. Creating new log.
I20260810 14:44:57.941547 32099 log.cc:824] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:44:57.944972 32099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d: No bootstrap required, opened a new log
I20260810 14:44:57.947026 32102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:57.933494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55008 (local address 127.24.215.190:36407)
0810 14:44:57.934596 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:57.934610 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:57.934658 (+ 48us) server_negotiation.cc:407] Connection header received
0810 14:44:57.937020 (+ 2362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:57.937048 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:57.937605 (+ 557us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:57.938046 (+ 441us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:44:57.938623 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:57.940628 (+ 2005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:57.942075 (+ 1447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:57.942862 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:57.943059 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:57.944224 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:44:57.944235 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:44:57.944253 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:44:57.944962 (+ 709us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:44:57.945066 (+ 104us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:44:57.945094 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:44:57.945216 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 14:44:57.945427 (+ 211us) server_negotiation.cc:299] Negotiation successful
0810 14:44:57.945775 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":357,"threads_started":1}
I20260810 14:44:57.951442 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:44:57.955343 32099 raft_consensus.cc:348] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f1cd31c08c4570a92465630675611d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 36407 } }
I20260810 14:44:57.955794 32099 raft_consensus.cc:374] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:44:57.955925 32099 raft_consensus.cc:729] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 52f1cd31c08c4570a92465630675611d, State: Initialized, Role: FOLLOWER
I20260810 14:44:57.956549 32099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f1cd31c08c4570a92465630675611d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 36407 } }
I20260810 14:44:57.956779 32099 raft_consensus.cc:388] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:44:57.956940 32099 raft_consensus.cc:482] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:44:57.957119 32099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:44:57.959555 32099 raft_consensus.cc:504] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f1cd31c08c4570a92465630675611d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 36407 } }
I20260810 14:44:57.960111 32099 leader_election.cc:302] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 52f1cd31c08c4570a92465630675611d; no voters:
I20260810 14:44:57.960736 32099 leader_election.cc:288] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:44:57.960899 32104 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:44:57.962304 32104 raft_consensus.cc:686] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [term 1 LEADER]: Becoming Leader. State: Replica: 52f1cd31c08c4570a92465630675611d, State: Running, Role: LEADER
I20260810 14:44:57.962877 32104 consensus_queue.cc:237] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f1cd31c08c4570a92465630675611d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 36407 } }
I20260810 14:44:57.963192 32099 sys_catalog.cc:564] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:44:57.965729 32106 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 52f1cd31c08c4570a92465630675611d. Latest consensus state: current_term: 1 leader_uuid: "52f1cd31c08c4570a92465630675611d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f1cd31c08c4570a92465630675611d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 36407 } } }
I20260810 14:44:57.966096 32106 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:57.966626 32105 sys_catalog.cc:455] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "52f1cd31c08c4570a92465630675611d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "52f1cd31c08c4570a92465630675611d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 36407 } } }
I20260810 14:44:57.966946 32105 sys_catalog.cc:458] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d [sys.catalog]: This master's current role is: LEADER
I20260810 14:44:57.968823 32115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:44:57.974509 32115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:44:57.984143 32115 catalog_manager.cc:1347] Generated new cluster ID: 708b35a8f5b040dd8d99e8310b981b14
I20260810 14:44:57.984272 32115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:44:57.997134 32115 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:44:57.998471 32115 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:44:58.005393 32115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 52f1cd31c08c4570a92465630675611d: Generated new TSK 0
I20260810 14:44:58.006245 32115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:44:58.075372 32102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:44:58.029072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55010 (local address 127.24.215.190:36407)
0810 14:44:58.029278 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 14:44:58.029285 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:44:58.029657 (+ 372us) server_negotiation.cc:407] Connection header received
0810 14:44:58.029875 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:44:58.029881 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:44:58.029993 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:44:58.030168 (+ 175us) server_negotiation.cc:226] Negotiated authn=JWT
0810 14:44:58.030565 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:58.031788 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:58.033361 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:44:58.034046 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:44:58.034171 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:58.034658 (+ 487us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0810 14:44:58.074512 (+ 39854us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 14:44:58.074985 (+ 473us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55010: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":76}
W20260810 14:44:58.075533 32102 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55010: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260810 14:44:58.075521 32133 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:44:58.028955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:36407 (local address 127.0.0.1:55010)
0810 14:44:58.029552 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0810 14:44:58.029568 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:44:58.029690 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:44:58.030178 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:44:58.030181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:44:58.030191 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0810 14:44:58.030416 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:58.030424 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:58.031973 (+ 1549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:58.031977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:58.033216 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:44:58.033229 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:44:58.034226 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:44:58.034230 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:44:58.034346 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:44:58.034379 (+ 33us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0810 14:44:58.074836 (+ 40457us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0810 14:44:58.075164 (+ 328us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.24.215.190:36407: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":315,"threads_started":1}
I20260810 14:44:58.076900 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 32023
I20260810 14:44:58.078210 32094 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:44:58.175137 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 32023
2026-08-10T14:44:58Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (754 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (1590 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (418 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:45:00 dist-test-slave-vkjp krb5kdc[32151](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:45:00 dist-test-slave-vkjp krb5kdc[32151](info): set up 2 sockets
Aug 10 14:45:00 dist-test-slave-vkjp krb5kdc[32151](info): commencing operation
krb5kdc: starting...
W20260810 14:45:01.361940 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:45:01 dist-test-slave-vkjp krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:45:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:45:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:45:01.603506 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:42465
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:40089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:42465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:01.938599 32169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:01.938957 32169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:01.939054 32169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:01.946758 32169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:45:01.946875 32169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:01.946928 32169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:01.946985 32169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:45:01.947029 32169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:45:01.957314 32169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:40089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:42465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:42465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:01.958667 32169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:01.960067 32169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:01.972219 32175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:01.972249 32176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:01.972298 32178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:01.973441 32169 server_base.cc:1034] running on GCE node
I20260810 14:45:01.974246 32169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:01.975787 32169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:01.977005 32169 hybrid_clock.cc:648] HybridClock initialized: now 1786373101976932 us; error 54 us; skew 500 ppm
I20260810 14:45:01.977412 32169 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:01 dist-test-slave-vkjp krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373101, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:01.986403 32169 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:45:01.988212 32169 webserver.cc:460] Webserver started at http://127.24.215.190:33247/ using document root <none> and password file <none>
I20260810 14:45:01.988946 32169 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:01.989056 32169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:01.989341 32169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:01.992640 32169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "97138bb5339d42c59658d5e1b8527a25"
format_stamp: "Formatted at 2026-08-10 14:45:01 on dist-test-slave-vkjp"
server_key: "23eaeb05320b3fe438f00eb65eb0e5f3"
server_key_iv: "0b1b265ee18561715740c010749e069f"
server_key_version: "encryptionkey@0"
I20260810 14:45:01.993446 32169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "97138bb5339d42c59658d5e1b8527a25"
format_stamp: "Formatted at 2026-08-10 14:45:01 on dist-test-slave-vkjp"
server_key: "23eaeb05320b3fe438f00eb65eb0e5f3"
server_key_iv: "0b1b265ee18561715740c010749e069f"
server_key_version: "encryptionkey@0"
I20260810 14:45:01.999019 32169 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 14:45:02.002764 32186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:02.004293 32169 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:45:02.004536 32169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "97138bb5339d42c59658d5e1b8527a25"
format_stamp: "Formatted at 2026-08-10 14:45:01 on dist-test-slave-vkjp"
server_key: "23eaeb05320b3fe438f00eb65eb0e5f3"
server_key_iv: "0b1b265ee18561715740c010749e069f"
server_key_version: "encryptionkey@0"
I20260810 14:45:02.004760 32169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:02.030122 32169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:02.034343 32169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:02.034657 32169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:02.054644 32169 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:42465
I20260810 14:45:02.054736 32244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:42465 every 8 connection(s)
I20260810 14:45:02.056102 32169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:45:02.056835 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 32169
I20260810 14:45:02.057091 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:45:02.057878 25438 external_mini_cluster.cc:1428] Setting key 09c0c12f182115ce12da249c749acfd9
I20260810 14:45:02.060752 32245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:02.072103 32245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25: Bootstrap starting.
Aug 10 14:45:02 dist-test-slave-vkjp krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373101, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:45:02.075963 32245 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:02.077132 32245 log.cc:824] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:02.080770 32245 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25: No bootstrap required, opened a new log
I20260810 14:45:02.082140 32251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:02.059995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52096 (local address 127.24.215.190:42465)
0810 14:45:02.060809 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:02.060824 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:02.060894 (+ 70us) server_negotiation.cc:407] Connection header received
0810 14:45:02.062715 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:02.062738 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:02.063269 (+ 531us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:02.063716 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:02.065285 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:02.066819 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:02.068222 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:02.068939 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:02.069163 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:02.073782 (+ 4619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:02.073794 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:02.073810 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:02.073857 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:02.077316 (+ 3459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:02.078260 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:02.078272 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:02.078284 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:02.078571 (+ 287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:02.079065 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:02.079068 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:02.079070 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:02.079717 (+ 647us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:02.079955 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:02.080549 (+ 594us) server_negotiation.cc:299] Negotiation successful
0810 14:45:02.081037 (+ 488us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":372,"threads_started":1}
I20260810 14:45:02.089075 32245 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97138bb5339d42c59658d5e1b8527a25" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 42465 } }
I20260810 14:45:02.089474 32245 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:02.089593 32245 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97138bb5339d42c59658d5e1b8527a25, State: Initialized, Role: FOLLOWER
I20260810 14:45:02.090216 32245 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97138bb5339d42c59658d5e1b8527a25" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 42465 } }
I20260810 14:45:02.090463 32245 raft_consensus.cc:388] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:45:02.090620 32245 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:45:02.090812 32245 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:02.093428 32245 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97138bb5339d42c59658d5e1b8527a25" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 42465 } }
I20260810 14:45:02.093951 32245 leader_election.cc:302] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 97138bb5339d42c59658d5e1b8527a25; no voters:
I20260810 14:45:02.094609 32245 leader_election.cc:288] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:45:02.094766 32253 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:45:02.096014 32253 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [term 1 LEADER]: Becoming Leader. State: Replica: 97138bb5339d42c59658d5e1b8527a25, State: Running, Role: LEADER
I20260810 14:45:02.096606 32253 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97138bb5339d42c59658d5e1b8527a25" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 42465 } }
I20260810 14:45:02.096918 32245 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:45:02.099557 32255 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97138bb5339d42c59658d5e1b8527a25. Latest consensus state: current_term: 1 leader_uuid: "97138bb5339d42c59658d5e1b8527a25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97138bb5339d42c59658d5e1b8527a25" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 42465 } } }
I20260810 14:45:02.099766 32254 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "97138bb5339d42c59658d5e1b8527a25" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "97138bb5339d42c59658d5e1b8527a25" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 42465 } } }
I20260810 14:45:02.100033 32255 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:02.100093 32254 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25 [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:02.101747 32262 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:45:02.106417 32262 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:45:02.115849 32262 catalog_manager.cc:1347] Generated new cluster ID: 59b5f48e04234599af7effc0d4da5de9
I20260810 14:45:02.116003 32262 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:45:02.135036 32262 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:45:02.137125 32262 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:45:02.155730 32262 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97138bb5339d42c59658d5e1b8527a25: Generated new TSK 0
I20260810 14:45:02.156849 32262 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:45:02.216005 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:42465
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:40089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:02.560343 32276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:02.560662 32276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:02.560844 32276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:02.568416 32276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:02.568572 32276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:02.568744 32276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:45:02.578706 32276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:40089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:42465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:02.580125 32276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:02.581537 32276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:02.594753 32281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:02.594789 32285 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:02.594702 32282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:02.596002 32276 server_base.cc:1034] running on GCE node
I20260810 14:45:02.596590 32276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:02.597581 32276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:02.598785 32276 hybrid_clock.cc:648] HybridClock initialized: now 1786373102598745 us; error 33 us; skew 500 ppm
I20260810 14:45:02.599071 32276 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:02 dist-test-slave-vkjp krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373102, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:02.602474 32276 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:45:02.604027 32276 webserver.cc:460] Webserver started at http://127.24.215.129:42055/ using document root <none> and password file <none>
I20260810 14:45:02.604723 32276 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:02.604858 32276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:02.605192 32276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:02.608281 32276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "1196cec7b628423db37efbf9c9624c8c"
format_stamp: "Formatted at 2026-08-10 14:45:02 on dist-test-slave-vkjp"
server_key: "9f7a18329987e13b702b2bcacda1288c"
server_key_iv: "e02d7cfe056fb159b1536a710bc06138"
server_key_version: "encryptionkey@0"
I20260810 14:45:02.609220 32276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "1196cec7b628423db37efbf9c9624c8c"
format_stamp: "Formatted at 2026-08-10 14:45:02 on dist-test-slave-vkjp"
server_key: "9f7a18329987e13b702b2bcacda1288c"
server_key_iv: "e02d7cfe056fb159b1536a710bc06138"
server_key_version: "encryptionkey@0"
I20260810 14:45:02.614490 32276 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 14:45:02.618031 32292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:02.619400 32276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:45:02.619567 32276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "1196cec7b628423db37efbf9c9624c8c"
format_stamp: "Formatted at 2026-08-10 14:45:02 on dist-test-slave-vkjp"
server_key: "9f7a18329987e13b702b2bcacda1288c"
server_key_iv: "e02d7cfe056fb159b1536a710bc06138"
server_key_version: "encryptionkey@0"
I20260810 14:45:02.619765 32276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:02.634029 32276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:02.638468 32276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:02.638770 32276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:02.639799 32276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:02.641148 32276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:02.641223 32276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:02.641350 32276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:02.641422 32276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:02.674114 32276 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:44607
I20260810 14:45:02.674162 32423 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:44607 every 8 connection(s)
I20260810 14:45:02.675554 32276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:45:02.681166 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 32276
I20260810 14:45:02.681418 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:45:02.682288 25438 external_mini_cluster.cc:1428] Setting key b5503218b3adcb115a0101e0e78b02a6
Aug 10 14:45:02 dist-test-slave-vkjp krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373102, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:45:02.695847 32428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:02.678110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:46159 (local address 127.24.215.190:42465)
0810 14:45:02.678593 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:02.678597 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:02.679479 (+ 882us) server_negotiation.cc:407] Connection header received
0810 14:45:02.681345 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:02.681364 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:02.681455 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:02.681589 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:02.683578 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:02.684353 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:02.685878 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:02.686308 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:02.686455 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:02.690135 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:02.690140 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:02.690144 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:02.690177 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:02.692771 (+ 2594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:02.693494 (+ 723us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:02.693504 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:02.693507 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:02.693623 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:02.694075 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:02.694080 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:02.694085 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:02.694352 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:02.694522 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:02.695246 (+ 724us) server_negotiation.cc:299] Negotiation successful
0810 14:45:02.695483 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":380,"thread_start_us":274,"threads_started":1}
I20260810 14:45:02.696763 32429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:02.678513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:42465 (local address 127.24.215.129:46159)
0810 14:45:02.679304 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:02.679349 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:02.681035 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:02.681858 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:02.681871 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:02.682656 (+ 785us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:02.683391 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:02.683418 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:02.684514 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:02.684521 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:02.685729 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:02.685741 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:02.686546 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:02.686554 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:02.686792 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:02.687496 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:02.687518 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:02.689886 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:02.692927 (+ 3041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:02.692937 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:02.692952 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:02.693360 (+ 408us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:02.693737 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:02.693742 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:02.693746 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:02.693950 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:02.694496 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:02.694503 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:02.694839 (+ 336us) client_negotiation.cc:769] Sending connection context
0810 14:45:02.695122 (+ 283us) client_negotiation.cc:240] Negotiation successful
0810 14:45:02.695638 (+ 516us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":292,"threads_started":1}
I20260810 14:45:02.698410 32425 heartbeater.cc:347] Connected to a master server at 127.24.215.190:42465
I20260810 14:45:02.698765 32425 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:02.699754 32425 heartbeater.cc:511] Master 127.24.215.190:42465 requested a full tablet report, sending...
I20260810 14:45:02.702266 32204 ts_manager.cc:194] Registered new tserver with Master: 1196cec7b628423db37efbf9c9624c8c (127.24.215.129:44607)
I20260810 14:45:02.703791 32204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:46159
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:45:02.733912 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:42465
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:40089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:03.065275 32432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:03.065590 32432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:03.065765 32432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:03.072893 32432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:03.073050 32432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:03.073237 32432 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:45:03.082777 32432 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:40089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:42465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:03.084080 32432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:03.085439 32432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:03.098376 32438 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:03.098399 32439 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:03.098332 32441 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:03.099663 32432 server_base.cc:1034] running on GCE node
I20260810 14:45:03.100260 32432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:03.101269 32432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:03.102492 32432 hybrid_clock.cc:648] HybridClock initialized: now 1786373103102441 us; error 46 us; skew 500 ppm
I20260810 14:45:03.102815 32432 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:03 dist-test-slave-vkjp krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373103, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:03.106071 32432 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:45:03.107606 32432 webserver.cc:460] Webserver started at http://127.24.215.130:41573/ using document root <none> and password file <none>
I20260810 14:45:03.108392 32432 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:03.108549 32432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:03.108861 32432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:03.111928 32432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "c3381649e9764ed6b8063f5a74bb41ba"
format_stamp: "Formatted at 2026-08-10 14:45:03 on dist-test-slave-vkjp"
server_key: "0d86ce8bf4fab4d192985485fd5d5190"
server_key_iv: "ec332159c9bce8fc9c9f92786c7f96c9"
server_key_version: "encryptionkey@0"
I20260810 14:45:03.112712 32432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "c3381649e9764ed6b8063f5a74bb41ba"
format_stamp: "Formatted at 2026-08-10 14:45:03 on dist-test-slave-vkjp"
server_key: "0d86ce8bf4fab4d192985485fd5d5190"
server_key_iv: "ec332159c9bce8fc9c9f92786c7f96c9"
server_key_version: "encryptionkey@0"
I20260810 14:45:03.117910 32432 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:45:03.121291 32450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:03.122550 32432 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 14:45:03.122783 32432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "c3381649e9764ed6b8063f5a74bb41ba"
format_stamp: "Formatted at 2026-08-10 14:45:03 on dist-test-slave-vkjp"
server_key: "0d86ce8bf4fab4d192985485fd5d5190"
server_key_iv: "ec332159c9bce8fc9c9f92786c7f96c9"
server_key_version: "encryptionkey@0"
I20260810 14:45:03.123057 32432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:03.154039 32432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:03.158474 32432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:03.158782 32432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:03.159789 32432 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:03.161134 32432 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:03.161242 32432 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:03.161406 32432 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:03.161481 32432 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:03.195776 32432 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:33713
I20260810 14:45:03.195827 32589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:33713 every 8 connection(s)
I20260810 14:45:03.197274 32432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:45:03.205171 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 32432
I20260810 14:45:03.205312 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:45:03.205788 25438 external_mini_cluster.cc:1428] Setting key 27ace4a1ded09efbb8b27eafd7777bba
Aug 10 14:45:03 dist-test-slave-vkjp krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373103, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:45:03.220446 32592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:03.200020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:34761 (local address 127.24.215.190:42465)
0810 14:45:03.200478 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:03.200483 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:03.201716 (+ 1233us) server_negotiation.cc:407] Connection header received
0810 14:45:03.203800 (+ 2084us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:03.203807 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:03.203907 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:03.204037 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:03.205690 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.206535 (+ 845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.208689 (+ 2154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.209561 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.209729 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:03.213799 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:03.213804 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:03.213807 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:03.213838 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:03.216654 (+ 2816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:03.217637 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:03.217646 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:03.217652 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:03.217812 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:03.218303 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:03.218311 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:03.218317 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:03.218697 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:03.218866 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:03.219635 (+ 769us) server_negotiation.cc:299] Negotiation successful
0810 14:45:03.219946 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":231,"threads_started":1}
I20260810 14:45:03.221067 32593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:03.200626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:42465 (local address 127.24.215.130:34761)
0810 14:45:03.201550 (+ 924us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:03.201598 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:03.203512 (+ 1914us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:03.204284 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:03.204292 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:03.204832 (+ 540us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:03.205508 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:03.205529 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.206701 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.206705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:03.208537 (+ 1832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:03.208551 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.209783 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.209788 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:03.210039 (+ 251us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:03.210727 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:03.210754 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:03.213567 (+ 2813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:03.216813 (+ 3246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:03.216832 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:03.216850 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:03.217501 (+ 651us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:03.217972 (+ 471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:03.217976 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:03.217978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:03.218173 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:03.218820 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:03.218826 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:03.219234 (+ 408us) client_negotiation.cc:769] Sending connection context
0810 14:45:03.219489 (+ 255us) client_negotiation.cc:240] Negotiation successful
0810 14:45:03.219906 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":668,"thread_start_us":313,"threads_started":1}
I20260810 14:45:03.222571 32590 heartbeater.cc:347] Connected to a master server at 127.24.215.190:42465
I20260810 14:45:03.222929 32590 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:03.223891 32590 heartbeater.cc:511] Master 127.24.215.190:42465 requested a full tablet report, sending...
I20260810 14:45:03.225562 32204 ts_manager.cc:194] Registered new tserver with Master: c3381649e9764ed6b8063f5a74bb41ba (127.24.215.130:33713)
I20260810 14:45:03.226572 32204 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:34761
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:45:03.258579 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:42465
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:40089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:03.592175 32596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:03.592540 32596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:03.592707 32596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:03.601287 32596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:03.601432 32596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:03.601598 32596 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:45:03.611817 32596 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:40089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:42465
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:03.613165 32596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:03.614488 32596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:03.628062 32603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:03.628265 32602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:03.628225 32605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:03.629444 32596 server_base.cc:1034] running on GCE node
I20260810 14:45:03.630065 32596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:03.631011 32596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:03.632268 32596 hybrid_clock.cc:648] HybridClock initialized: now 1786373103632230 us; error 31 us; skew 500 ppm
I20260810 14:45:03.632702 32596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:03 dist-test-slave-vkjp krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373103, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:03.635960 32596 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:45:03.637548 32596 webserver.cc:460] Webserver started at http://127.24.215.131:40963/ using document root <none> and password file <none>
I20260810 14:45:03.638269 32596 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:03.638423 32596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:03.638742 32596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:03.641938 32596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "49f70e8748fb41a49911ec38822991df"
format_stamp: "Formatted at 2026-08-10 14:45:03 on dist-test-slave-vkjp"
server_key: "24fa820e4dddd7f848d36165e7bcfc2b"
server_key_iv: "bbc2adc3ef0d5a58784a3e0b683875af"
server_key_version: "encryptionkey@0"
I20260810 14:45:03.642773 32596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "49f70e8748fb41a49911ec38822991df"
format_stamp: "Formatted at 2026-08-10 14:45:03 on dist-test-slave-vkjp"
server_key: "24fa820e4dddd7f848d36165e7bcfc2b"
server_key_iv: "bbc2adc3ef0d5a58784a3e0b683875af"
server_key_version: "encryptionkey@0"
I20260810 14:45:03.648216 32596 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 14:45:03.651710 32613 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:03.653018 32596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:45:03.653198 32596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "49f70e8748fb41a49911ec38822991df"
format_stamp: "Formatted at 2026-08-10 14:45:03 on dist-test-slave-vkjp"
server_key: "24fa820e4dddd7f848d36165e7bcfc2b"
server_key_iv: "bbc2adc3ef0d5a58784a3e0b683875af"
server_key_version: "encryptionkey@0"
I20260810 14:45:03.653414 32596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:03.672086 32596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:03.676434 32596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:03.676718 32596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:03.677721 32596 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:03.678992 32596 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:03.679092 32596 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:03.679286 32596 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:03.679358 32596 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:03.707015 32425 heartbeater.cc:503] Master 127.24.215.190:42465 was elected leader, sending a full tablet report...
I20260810 14:45:03.712800 32596 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:40825
I20260810 14:45:03.712836 32748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:40825 every 8 connection(s)
I20260810 14:45:03.714094 32596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:45:03.719816 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 32596
I20260810 14:45:03.719986 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:45:03.720562 25438 external_mini_cluster.cc:1428] Setting key 0ed0a82467f7fdd262f94b4fcd96d601
Aug 10 14:45:03 dist-test-slave-vkjp krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373103, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:45:03.734069 32592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:03.716719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:34661 (local address 127.24.215.190:42465)
0810 14:45:03.716894 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:03.716900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:03.718186 (+ 1286us) server_negotiation.cc:407] Connection header received
0810 14:45:03.720199 (+ 2013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:03.720205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:03.720310 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:03.720426 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:03.722120 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.723120 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.725012 (+ 1892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.725415 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.725523 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:03.728966 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:03.728971 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:03.728974 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:03.729007 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:03.731476 (+ 2469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:03.732120 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:03.732123 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:03.732126 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:03.732240 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:03.732603 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:03.732606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:03.732608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:03.732905 (+ 297us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:03.733019 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:03.733649 (+ 630us) server_negotiation.cc:299] Negotiation successful
0810 14:45:03.733817 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:45:03.735262 32751 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:03.717214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:42465 (local address 127.24.215.131:34661)
0810 14:45:03.718020 (+ 806us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:03.718061 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:03.719829 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:03.720668 (+ 839us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:03.720679 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:03.721242 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:03.721952 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:03.721973 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.723294 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.723299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:03.724852 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:03.724865 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.725604 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.725609 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:03.725797 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:03.726491 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:03.726519 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:03.728775 (+ 2256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:03.731598 (+ 2823us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:03.731605 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:03.731616 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:03.732021 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:03.732333 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:03.732336 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:03.732339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:03.732508 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:03.733008 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:03.733013 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:03.733318 (+ 305us) client_negotiation.cc:769] Sending connection context
0810 14:45:03.733544 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 14:45:03.733914 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":295,"threads_started":1}
I20260810 14:45:03.736644 32749 heartbeater.cc:347] Connected to a master server at 127.24.215.190:42465
I20260810 14:45:03.736974 32749 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:03.737681 32749 heartbeater.cc:511] Master 127.24.215.190:42465 requested a full tablet report, sending...
I20260810 14:45:03.739158 32203 ts_manager.cc:194] Registered new tserver with Master: 49f70e8748fb41a49911ec38822991df (127.24.215.131:40825)
I20260810 14:45:03.740023 32203 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:34661
I20260810 14:45:03.748905 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/test-admin.keytab.
Aug 10 14:45:03 dist-test-slave-vkjp krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:03.777971 25438 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260810 14:45:03.783787 32756 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:03.780857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38152 (local address 127.24.215.129:44607)
0810 14:45:03.781646 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:03.781654 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:03.781680 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:45:03.781742 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:03.781749 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:03.781967 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:03.782139 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:03.783318 (+ 1179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38152: BlockingRecv error: recv got EOF from 127.0.0.1:38152 (error 108)
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":398,"threads_started":1}
W20260810 14:45:03.783880 25438 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/krb5cc' of type FILE
Aug 10 14:45:03 dist-test-slave-vkjp krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:03.785619 25438 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 14:45:03 dist-test-slave-vkjp krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:03.787014 25438 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 14:45:03 dist-test-slave-vkjp krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:45:03.803150 32756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:03.788713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38156 (local address 127.24.215.129:44607)
0810 14:45:03.788915 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:03.788921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:03.789069 (+ 148us) server_negotiation.cc:407] Connection header received
0810 14:45:03.789294 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:03.789298 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:03.789357 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:03.789458 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:03.790512 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.791710 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.792958 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.793539 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.793692 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:03.797265 (+ 3573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:03.797277 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:03.797292 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:03.797355 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:03.800367 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:03.801008 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:03.801014 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:03.801017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:03.801332 (+ 315us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:03.801714 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:03.801720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:03.801723 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:03.802085 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:03.802334 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:03.802557 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 14:45:03.802816 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
W20260810 14:45:03.808835 32756 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:03.807251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38162 (local address 127.24.215.129:44607)
0810 14:45:03.807482 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:03.807488 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:03.807663 (+ 175us) server_negotiation.cc:407] Connection header received
0810 14:45:03.807781 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:03.807785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:03.807852 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:03.807936 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:03.808618 (+ 682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38162: BlockingRecv error: recv got EOF from 127.0.0.1:38162 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260810 14:45:03.809074 25438 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/krb5cc' of type FILE
Aug 10 14:45:03 dist-test-slave-vkjp krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:03.810660 25438 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 14:45:03 dist-test-slave-vkjp krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:03.812031 25438 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 14:45:03 dist-test-slave-vkjp krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:45:03.825693 32756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:03.813236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38178 (local address 127.24.215.129:44607)
0810 14:45:03.813435 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:03.813443 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:03.813716 (+ 273us) server_negotiation.cc:407] Connection header received
0810 14:45:03.813811 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:03.813814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:03.813865 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:03.813946 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:03.814814 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.815630 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.816782 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.817216 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.817318 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:03.820720 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:03.820725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:03.820728 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:03.820771 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:03.823280 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:03.823928 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:03.823931 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:03.823934 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:03.824049 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:03.824427 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:03.824430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:03.824432 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:03.824748 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:03.824955 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:03.825225 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 14:45:03.825411 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260810 14:45:03.830374 32756 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:03.828654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38188 (local address 127.24.215.129:44607)
0810 14:45:03.828855 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:03.828862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:03.829135 (+ 273us) server_negotiation.cc:407] Connection header received
0810 14:45:03.829309 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:03.829321 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:03.829393 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:03.829492 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:03.830174 (+ 682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38188: BlockingRecv error: recv got EOF from 127.0.0.1:38188 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 14:45:03.830646 25438 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786373051600012-25438-0/krb5kdc/krb5cc' of type FILE
Aug 10 14:45:03 dist-test-slave-vkjp krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:03.832082 25438 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 10 14:45:03 dist-test-slave-vkjp krb5kdc[32151](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:03.833297 25438 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 10 14:45:03 dist-test-slave-vkjp krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:45:03.846828 32756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:03.834500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38200 (local address 127.24.215.129:44607)
0810 14:45:03.834728 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:03.834734 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:03.834960 (+ 226us) server_negotiation.cc:407] Connection header received
0810 14:45:03.835063 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:03.835067 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:03.835178 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:03.835255 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:03.836084 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.836798 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.837936 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.838342 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.838460 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:03.841791 (+ 3331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:03.841796 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:03.841800 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:03.841830 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:03.844504 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:03.845106 (+ 602us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:03.845110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:03.845112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:03.845234 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:03.845531 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:03.845535 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:03.845536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:03.845892 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:03.846099 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:03.846335 (+ 236us) server_negotiation.cc:299] Negotiation successful
0810 14:45:03.846537 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260810 14:45:03.848405 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 32276
I20260810 14:45:03.859994 32756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:03.849194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38206 (local address 127.24.215.129:44607)
0810 14:45:03.849443 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:03.849448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:03.849586 (+ 138us) server_negotiation.cc:407] Connection header received
0810 14:45:03.849701 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:03.849704 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:03.849754 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:03.849825 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:03.850656 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.851418 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.852540 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.852974 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.853066 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:03.855200 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:03.855206 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:03.855209 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:03.855245 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:03.857588 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:03.858233 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:03.858237 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:03.858239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:03.858391 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:03.858713 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:03.858717 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:03.858719 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:03.858956 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:03.859208 (+ 252us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:03.859474 (+ 266us) server_negotiation.cc:299] Negotiation successful
0810 14:45:03.859664 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 14:45:03.860651 32419 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:03.944208 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 32276
I20260810 14:45:03.965444 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 32432
Aug 10 14:45:03 dist-test-slave-vkjp krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:45:03.980965 303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:03.966323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45518 (local address 127.24.215.130:33713)
0810 14:45:03.966855 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:03.966862 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:03.966898 (+ 36us) server_negotiation.cc:407] Connection header received
0810 14:45:03.966965 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:03.966972 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:03.967238 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:03.967421 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:03.968406 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.969414 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.970701 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:03.971343 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:03.971492 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:03.975471 (+ 3979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:03.975482 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:03.975496 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:03.975529 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:03.978364 (+ 2835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:03.979161 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:03.979168 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:03.979170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:03.979305 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:03.979613 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:03.979617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:03.979618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:03.979953 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:03.980166 (+ 213us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:03.980454 (+ 288us) server_negotiation.cc:299] Negotiation successful
0810 14:45:03.980644 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":403,"thread_start_us":278,"threads_started":1}
I20260810 14:45:03.982239 32584 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:04.064872 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 32432
I20260810 14:45:04.085132 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 32596
Aug 10 14:45:04 dist-test-slave-vkjp krb5kdc[32151](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373103, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:45:04.101608 306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:04.086298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60114 (local address 127.24.215.131:40825)
0810 14:45:04.087031 (+ 733us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:04.087038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:04.087067 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:45:04.087194 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:04.087201 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:04.087420 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:04.087557 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:04.088545 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:04.089571 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:04.090757 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:04.091433 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:04.091531 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:04.095508 (+ 3977us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:04.095519 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:04.095533 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:04.095570 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:04.098771 (+ 3201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:04.099492 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:04.099499 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:04.099501 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:04.099645 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:04.100006 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:04.100010 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:04.100011 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:04.100428 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:04.100665 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:04.100934 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 14:45:04.101269 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":384,"threads_started":1}
I20260810 14:45:04.103302 32744 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:04.186131 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 32596
I20260810 14:45:04.205960 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 32169
I20260810 14:45:04.207367 32240 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:04.311362 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 32169
2026-08-10T14:45:04Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4131 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:45:04 dist-test-slave-vkjp krb5kdc[313](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:45:04 dist-test-slave-vkjp krb5kdc[313](info): set up 2 sockets
Aug 10 14:45:04 dist-test-slave-vkjp krb5kdc[313](info): commencing operation
krb5kdc: starting...
W20260810 14:45:05.504689 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.003s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:45:05 dist-test-slave-vkjp krb5kdc[313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:45:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:45:05Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.190.keytab.
Entry for principal oryx/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.190.keytab.
I20260810 14:45:05.746236 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:41267
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.190.keytab
--principal=oryx/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:41267
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:06.082244 326 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:06.082541 326 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:06.082633 326 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:06.089757 326 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:45:06.089854 326 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:06.089900 326 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:06.089972 326 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:45:06.090009 326 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:45:06.099674 326 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:41267
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.190.keytab
--principal=oryx/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:41267
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:06.100864 326 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:06.102131 326 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:06.113976 331 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:06.114022 332 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:06.114264 334 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:06.115017 326 server_base.cc:1034] running on GCE node
I20260810 14:45:06.115825 326 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:06.117225 326 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:06.118479 326 hybrid_clock.cc:648] HybridClock initialized: now 1786373106118415 us; error 46 us; skew 500 ppm
I20260810 14:45:06.118831 326 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:06 dist-test-slave-vkjp krb5kdc[313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373106, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:06.121930 326 init.cc:377] Logged in from keytab as oryx/127.24.215.190@KRBTEST.COM (short username oryx)
I20260810 14:45:06.123533 326 webserver.cc:460] Webserver started at http://127.24.215.190:36983/ using document root <none> and password file <none>
I20260810 14:45:06.124217 326 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:06.124362 326 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:06.124671 326 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:06.127664 326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "c5cde194e5e348ceac077b3a924912ec"
format_stamp: "Formatted at 2026-08-10 14:45:06 on dist-test-slave-vkjp"
server_key: "69bac6811c9568500f1c661e417b94dc"
server_key_iv: "ccec7b536e93b769287f72980605be2f"
server_key_version: "encryptionkey@0"
I20260810 14:45:06.128448 326 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "c5cde194e5e348ceac077b3a924912ec"
format_stamp: "Formatted at 2026-08-10 14:45:06 on dist-test-slave-vkjp"
server_key: "69bac6811c9568500f1c661e417b94dc"
server_key_iv: "ccec7b536e93b769287f72980605be2f"
server_key_version: "encryptionkey@0"
I20260810 14:45:06.133556 326 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:45:06.136853 343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:06.138203 326 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 14:45:06.138429 326 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "c5cde194e5e348ceac077b3a924912ec"
format_stamp: "Formatted at 2026-08-10 14:45:06 on dist-test-slave-vkjp"
server_key: "69bac6811c9568500f1c661e417b94dc"
server_key_iv: "ccec7b536e93b769287f72980605be2f"
server_key_version: "encryptionkey@0"
I20260810 14:45:06.138648 326 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:06.151938 326 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:06.156031 326 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:06.156368 326 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:06.176627 326 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:41267
I20260810 14:45:06.176651 408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:41267 every 8 connection(s)
I20260810 14:45:06.178166 326 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:45:06.179392 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 326
I20260810 14:45:06.179639 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:45:06.180487 25438 external_mini_cluster.cc:1428] Setting key 4390ecab36bf427a25364c346b51bef6
I20260810 14:45:06.182976 410 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 14:45:06 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM
I20260810 14:45:06.196830 410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec: Bootstrap starting.
I20260810 14:45:06.201241 410 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:06.202463 410 log.cc:824] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:06.205437 413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:06.182708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33902 (local address 127.24.215.190:41267)
0810 14:45:06.183633 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:06.183645 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:06.183689 (+ 44us) server_negotiation.cc:407] Connection header received
0810 14:45:06.185523 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:06.185545 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:06.185935 (+ 390us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:06.186266 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:06.188214 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:06.189608 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:06.191414 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:06.192074 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:06.192208 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:06.196845 (+ 4637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:06.196859 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:06.196879 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:06.196927 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:06.201304 (+ 4377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:06.202019 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:06.202030 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:06.202038 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:06.202259 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:06.202665 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:06.202669 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:06.202672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:06.203429 (+ 757us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:06.203771 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:06.204096 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 14:45:06.204476 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":735,"thread_start_us":423,"threads_started":1}
I20260810 14:45:06.205680 410 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec: No bootstrap required, opened a new log
I20260810 14:45:06.213155 410 raft_consensus.cc:348] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } }
I20260810 14:45:06.213577 410 raft_consensus.cc:374] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:06.213690 410 raft_consensus.cc:729] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c5cde194e5e348ceac077b3a924912ec, State: Initialized, Role: FOLLOWER
I20260810 14:45:06.214257 410 consensus_queue.cc:260] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } }
I20260810 14:45:06.214514 410 raft_consensus.cc:388] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:45:06.214680 410 raft_consensus.cc:482] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:45:06.214923 410 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:06.217576 410 raft_consensus.cc:504] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } }
I20260810 14:45:06.218101 410 leader_election.cc:302] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c5cde194e5e348ceac077b3a924912ec; no voters:
I20260810 14:45:06.218631 410 leader_election.cc:288] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:45:06.218777 415 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:45:06.219553 415 raft_consensus.cc:686] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [term 1 LEADER]: Becoming Leader. State: Replica: c5cde194e5e348ceac077b3a924912ec, State: Running, Role: LEADER
I20260810 14:45:06.220119 415 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } }
I20260810 14:45:06.220885 410 sys_catalog.cc:564] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:45:06.223172 418 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5cde194e5e348ceac077b3a924912ec. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } }
I20260810 14:45:06.223655 418 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:06.223757 416 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } }
I20260810 14:45:06.224189 416 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:06.226948 425 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:45:06.232090 425 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:45:06.240841 425 catalog_manager.cc:1347] Generated new cluster ID: 0602c91b97ca4eb48e7748d24ec2bf1b
I20260810 14:45:06.240952 425 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:45:06.269714 425 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:45:06.271061 425 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:45:06.279838 425 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec: Generated new TSK 0
I20260810 14:45:06.280647 425 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.129.keytab.
Entry for principal oryx/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.129.keytab.
I20260810 14:45:06.338222 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:41267
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.129.keytab
--principal=oryx/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:06.676441 441 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:06.676718 441 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:06.676878 441 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:06.684118 441 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:06.684226 441 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:06.684438 441 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:45:06.693982 441 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.129.keytab
--principal=oryx/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:41267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:06.695340 441 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:06.696718 441 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:06.710269 447 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:06.710296 446 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:06.710451 449 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:06.711143 441 server_base.cc:1034] running on GCE node
I20260810 14:45:06.711719 441 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:06.712610 441 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:06.713814 441 hybrid_clock.cc:648] HybridClock initialized: now 1786373106713778 us; error 33 us; skew 500 ppm
I20260810 14:45:06.714095 441 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:06 dist-test-slave-vkjp krb5kdc[313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373106, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:06.717373 441 init.cc:377] Logged in from keytab as oryx/127.24.215.129@KRBTEST.COM (short username oryx)
I20260810 14:45:06.718797 441 webserver.cc:460] Webserver started at http://127.24.215.129:45483/ using document root <none> and password file <none>
I20260810 14:45:06.719589 441 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:06.719743 441 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:06.720062 441 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:06.723312 441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "770eff9431a14800b13cbb540c4483a6"
format_stamp: "Formatted at 2026-08-10 14:45:06 on dist-test-slave-vkjp"
server_key: "aaf54fa57eddb5aa0a65e78ee50c120f"
server_key_iv: "54006ca97de146ebd715aed66bf08ecd"
server_key_version: "encryptionkey@0"
I20260810 14:45:06.724151 441 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "770eff9431a14800b13cbb540c4483a6"
format_stamp: "Formatted at 2026-08-10 14:45:06 on dist-test-slave-vkjp"
server_key: "aaf54fa57eddb5aa0a65e78ee50c120f"
server_key_iv: "54006ca97de146ebd715aed66bf08ecd"
server_key_version: "encryptionkey@0"
I20260810 14:45:06.729799 441 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:45:06.733270 456 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:06.734527 441 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:45:06.734704 441 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "770eff9431a14800b13cbb540c4483a6"
format_stamp: "Formatted at 2026-08-10 14:45:06 on dist-test-slave-vkjp"
server_key: "aaf54fa57eddb5aa0a65e78ee50c120f"
server_key_iv: "54006ca97de146ebd715aed66bf08ecd"
server_key_version: "encryptionkey@0"
I20260810 14:45:06.734920 441 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:06.761272 441 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:06.765743 441 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:06.766068 441 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:06.767050 441 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:06.768453 441 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:06.768560 441 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:06.768718 441 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:06.768795 441 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:06.802546 441 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:46583
I20260810 14:45:06.802579 590 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:46583 every 8 connection(s)
I20260810 14:45:06.803925 441 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:45:06.812605 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 441
I20260810 14:45:06.812857 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:45:06.813568 25438 external_mini_cluster.cc:1428] Setting key 80df658f54f79f80204fcda4cf263825
Aug 10 14:45:06 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373106, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM
I20260810 14:45:06.826640 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:06.806624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:45537 (local address 127.24.215.190:41267)
0810 14:45:06.807126 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:06.807133 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:06.808044 (+ 911us) server_negotiation.cc:407] Connection header received
0810 14:45:06.809862 (+ 1818us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:06.809867 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:06.809965 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:06.810086 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:06.811611 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:06.812556 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:06.814655 (+ 2099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:06.815125 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:06.815238 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:06.819304 (+ 4066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:06.819314 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:06.819321 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:06.819366 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:06.822968 (+ 3602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:06.823784 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:06.823793 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:06.823800 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:06.823969 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:06.824510 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:06.824513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:06.824515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:06.824773 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:06.824884 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:06.825881 (+ 997us) server_negotiation.cc:299] Negotiation successful
0810 14:45:06.826138 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":230,"threads_started":1}
I20260810 14:45:06.827400 595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:06.807196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.129:45537)
0810 14:45:06.807894 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:06.807931 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:06.809593 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:06.810320 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:06.810328 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:06.810818 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:06.811440 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:06.811458 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:06.812804 (+ 1346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:06.812810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:06.814495 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:06.814510 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:06.815401 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:06.815411 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:06.815759 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:06.816578 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:06.816601 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:06.819017 (+ 2416us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:06.823186 (+ 4169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:06.823193 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:06.823207 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:06.823643 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:06.824108 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:06.824114 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:06.824120 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:06.824390 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:06.824917 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:06.824927 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:06.825408 (+ 481us) client_negotiation.cc:769] Sending connection context
0810 14:45:06.825718 (+ 310us) client_negotiation.cc:240] Negotiation successful
0810 14:45:06.826126 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":277,"threads_started":1}
I20260810 14:45:06.829080 592 heartbeater.cc:347] Connected to a master server at 127.24.215.190:41267
I20260810 14:45:06.829421 592 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:06.830358 592 heartbeater.cc:511] Master 127.24.215.190:41267 requested a full tablet report, sending...
I20260810 14:45:06.832952 363 ts_manager.cc:194] Registered new tserver with Master: 770eff9431a14800b13cbb540c4483a6 (127.24.215.129:46583)
I20260810 14:45:06.835230 363 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:45537
WARNING: no policy specified for oryx/127.24.215.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.215.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.130.keytab.
Entry for principal oryx/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.130.keytab.
I20260810 14:45:06.868623 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:41267
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.130.keytab
--principal=oryx/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:07.192646 598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:07.192905 598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:07.193023 598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:07.200251 598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:07.200363 598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:07.200479 598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:45:07.210153 598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.130.keytab
--principal=oryx/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:41267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:07.211478 598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:07.212770 598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:07.226284 603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:07.226349 605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:07.226311 607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:07.227243 598 server_base.cc:1034] running on GCE node
I20260810 14:45:07.227936 598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:07.228999 598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:07.230228 598 hybrid_clock.cc:648] HybridClock initialized: now 1786373107230184 us; error 59 us; skew 500 ppm
I20260810 14:45:07.230612 598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:07 dist-test-slave-vkjp krb5kdc[313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373107, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:07.240135 598 init.cc:377] Logged in from keytab as oryx/127.24.215.130@KRBTEST.COM (short username oryx)
I20260810 14:45:07.241932 598 webserver.cc:460] Webserver started at http://127.24.215.130:43075/ using document root <none> and password file <none>
I20260810 14:45:07.242694 598 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:07.242874 598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:07.243280 598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:07.246340 598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "05be3da9c7e74b218e6025789e323cb9"
format_stamp: "Formatted at 2026-08-10 14:45:07 on dist-test-slave-vkjp"
server_key: "5ef73ddcb6694420740d70fee353f658"
server_key_iv: "af4f8e3714250ec22e0ea1c172bf3b3e"
server_key_version: "encryptionkey@0"
I20260810 14:45:07.247238 598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "05be3da9c7e74b218e6025789e323cb9"
format_stamp: "Formatted at 2026-08-10 14:45:07 on dist-test-slave-vkjp"
server_key: "5ef73ddcb6694420740d70fee353f658"
server_key_iv: "af4f8e3714250ec22e0ea1c172bf3b3e"
server_key_version: "encryptionkey@0"
I20260810 14:45:07.252686 598 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 14:45:07.256362 614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:07.257738 598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:45:07.257917 598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "05be3da9c7e74b218e6025789e323cb9"
format_stamp: "Formatted at 2026-08-10 14:45:07 on dist-test-slave-vkjp"
server_key: "5ef73ddcb6694420740d70fee353f658"
server_key_iv: "af4f8e3714250ec22e0ea1c172bf3b3e"
server_key_version: "encryptionkey@0"
I20260810 14:45:07.258126 598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:07.275401 598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:07.279376 598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:07.279678 598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:07.280943 598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:07.282401 598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:07.282506 598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:07.282657 598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:07.282732 598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:07.319394 598 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:35709
I20260810 14:45:07.319438 743 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:35709 every 8 connection(s)
I20260810 14:45:07.320717 598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:45:07.330811 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 598
I20260810 14:45:07.331019 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:45:07.331494 25438 external_mini_cluster.cc:1428] Setting key 74dd17f69c436e0a5e275ad4c979dc72
Aug 10 14:45:07 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373107, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM
I20260810 14:45:07.343166 594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:07.323542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:44339 (local address 127.24.215.190:41267)
0810 14:45:07.323729 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:07.323733 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:07.325012 (+ 1279us) server_negotiation.cc:407] Connection header received
0810 14:45:07.326988 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:07.326995 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:07.327147 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:07.327297 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:07.328953 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:07.329699 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:07.331464 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:07.332017 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:07.332170 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:07.336614 (+ 4444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:07.336620 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:07.336624 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:07.336655 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:07.339751 (+ 3096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:07.340702 (+ 951us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:07.340705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:07.340708 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:07.340831 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:07.341329 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:07.341332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:07.341334 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:07.341602 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:07.341758 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:07.342467 (+ 709us) server_negotiation.cc:299] Negotiation successful
0810 14:45:07.342720 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:45:07.344115 747 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:07.324042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.130:44339)
0810 14:45:07.324819 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:07.324872 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:07.326675 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:07.327538 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:07.327546 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:07.328073 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:07.328762 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:07.328789 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:07.329886 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:07.329891 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:07.331306 (+ 1415us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:07.331320 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:07.332263 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:07.332271 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:07.332585 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:07.333310 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:07.333346 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:07.336358 (+ 3012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:07.339924 (+ 3566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:07.339935 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:07.339953 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:07.340580 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:07.340940 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:07.340948 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:07.340954 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:07.341216 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:07.341740 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:07.341750 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:07.342093 (+ 343us) client_negotiation.cc:769] Sending connection context
0810 14:45:07.342325 (+ 232us) client_negotiation.cc:240] Negotiation successful
0810 14:45:07.342860 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":305,"threads_started":1}
I20260810 14:45:07.345572 745 heartbeater.cc:347] Connected to a master server at 127.24.215.190:41267
I20260810 14:45:07.345896 745 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:07.346803 745 heartbeater.cc:511] Master 127.24.215.190:41267 requested a full tablet report, sending...
I20260810 14:45:07.348489 363 ts_manager.cc:194] Registered new tserver with Master: 05be3da9c7e74b218e6025789e323cb9 (127.24.215.130:35709)
I20260810 14:45:07.349344 363 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:44339
WARNING: no policy specified for oryx/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.131.keytab.
Entry for principal oryx/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.131.keytab.
I20260810 14:45:07.386351 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:41267
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.131.keytab
--principal=oryx/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:07.718111 750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:07.718438 750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:07.718586 750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:07.726056 750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:07.726212 750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:07.726384 750 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:45:07.736409 750 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.131.keytab
--principal=oryx/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:41267
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:07.737815 750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:07.739202 750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:07.752900 757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:07.752980 756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:07.752944 759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:07.753625 750 server_base.cc:1034] running on GCE node
I20260810 14:45:07.754204 750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:07.755242 750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:07.756471 750 hybrid_clock.cc:648] HybridClock initialized: now 1786373107756429 us; error 54 us; skew 500 ppm
I20260810 14:45:07.756817 750 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:07 dist-test-slave-vkjp krb5kdc[313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373107, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:07.760076 750 init.cc:377] Logged in from keytab as oryx/127.24.215.131@KRBTEST.COM (short username oryx)
I20260810 14:45:07.761601 750 webserver.cc:460] Webserver started at http://127.24.215.131:45497/ using document root <none> and password file <none>
I20260810 14:45:07.762310 750 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:07.762588 750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:07.762955 750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:07.766256 750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "b522cc31bd6141558da804220a6775fd"
format_stamp: "Formatted at 2026-08-10 14:45:07 on dist-test-slave-vkjp"
server_key: "f02b62df140e68da799724cbaf005260"
server_key_iv: "96541e541211b71286cc8db59f7924b6"
server_key_version: "encryptionkey@0"
I20260810 14:45:07.767243 750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "b522cc31bd6141558da804220a6775fd"
format_stamp: "Formatted at 2026-08-10 14:45:07 on dist-test-slave-vkjp"
server_key: "f02b62df140e68da799724cbaf005260"
server_key_iv: "96541e541211b71286cc8db59f7924b6"
server_key_version: "encryptionkey@0"
I20260810 14:45:07.772840 750 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:45:07.776325 766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:07.777675 750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:45:07.777861 750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "b522cc31bd6141558da804220a6775fd"
format_stamp: "Formatted at 2026-08-10 14:45:07 on dist-test-slave-vkjp"
server_key: "f02b62df140e68da799724cbaf005260"
server_key_iv: "96541e541211b71286cc8db59f7924b6"
server_key_version: "encryptionkey@0"
I20260810 14:45:07.778085 750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:07.797827 750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:07.802417 750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:07.802723 750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:07.803767 750 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:07.805174 750 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:07.805279 750 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:07.805429 750 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:07.805505 750 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:07.839288 592 heartbeater.cc:503] Master 127.24.215.190:41267 was elected leader, sending a full tablet report...
I20260810 14:45:07.839916 750 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:34477
I20260810 14:45:07.839980 904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:34477 every 8 connection(s)
I20260810 14:45:07.841627 750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:45:07.848027 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 750
I20260810 14:45:07.848246 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:45:07.848755 25438 external_mini_cluster.cc:1428] Setting key da0148f53e2442f053bd0ee1852a784a
Aug 10 14:45:07 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373107, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM
I20260810 14:45:07.863950 907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:07.844470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:54927 (local address 127.24.215.190:41267)
0810 14:45:07.844963 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:07.844968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:07.845909 (+ 941us) server_negotiation.cc:407] Connection header received
0810 14:45:07.847808 (+ 1899us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:07.847813 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:07.847903 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:07.848025 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:07.849756 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:07.850598 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:07.852434 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:07.852863 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:07.852964 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:07.856827 (+ 3863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:07.856835 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:07.856840 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:07.856880 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:07.860569 (+ 3689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:07.861314 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:07.861321 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:07.861325 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:07.861481 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:07.861885 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:07.861888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:07.861890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:07.862251 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:07.862402 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:07.863257 (+ 855us) server_negotiation.cc:299] Negotiation successful
0810 14:45:07.863544 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":245,"threads_started":1}
I20260810 14:45:07.864938 908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:07.845069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.131:54927)
0810 14:45:07.845757 (+ 688us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:07.845795 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:07.847541 (+ 1746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:07.848279 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:07.848288 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:07.848832 (+ 544us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:07.849585 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:07.849606 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:07.850754 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:07.850759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:07.852270 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:07.852287 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:07.853062 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:07.853067 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:07.853264 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:07.853900 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:07.853927 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:07.856583 (+ 2656us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:07.860713 (+ 4130us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:07.860720 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:07.860734 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:07.861182 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:07.861578 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:07.861582 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:07.861584 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:07.861781 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:07.862386 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:07.862394 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:07.862824 (+ 430us) client_negotiation.cc:769] Sending connection context
0810 14:45:07.863154 (+ 330us) client_negotiation.cc:240] Negotiation successful
0810 14:45:07.863667 (+ 513us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":261,"threads_started":1}
I20260810 14:45:07.866379 905 heartbeater.cc:347] Connected to a master server at 127.24.215.190:41267
I20260810 14:45:07.866706 905 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:07.867619 905 heartbeater.cc:511] Master 127.24.215.190:41267 requested a full tablet report, sending...
I20260810 14:45:07.869051 363 ts_manager.cc:194] Registered new tserver with Master: b522cc31bd6141558da804220a6775fd (127.24.215.131:34477)
I20260810 14:45:07.869910 363 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:54927
I20260810 14:45:07.876995 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.215.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.215.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.215.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.189.keytab.
Entry for principal oryx/127.24.215.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.189.keytab.
I20260810 14:45:07.930944 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.189:36703
--webserver_interface=127.24.215.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.189.keytab
--principal=oryx/127.24.215.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:41267,127.24.215.189:36703
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:08.268070 911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:08.268357 911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:08.268551 911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:08.275856 911 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:45:08.275970 911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:08.276083 911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:08.276185 911 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:45:08.276270 911 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:45:08.286455 911 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:41267,127.24.215.189:36703
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.189.keytab
--principal=oryx/127.24.215.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.189:36703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.215.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
W20260810 14:45:08.286844 911 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.24.215.190:41267,127.24.215.189:36703'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260810 14:45:08.287958 911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:08.289374 911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:08.301492 918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:08.301656 920 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:08.301630 924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:08.302524 911 server_base.cc:1034] running on GCE node
I20260810 14:45:08.303377 911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:08.304752 911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:08.305982 911 hybrid_clock.cc:648] HybridClock initialized: now 1786373108305915 us; error 45 us; skew 500 ppm
I20260810 14:45:08.306277 911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:08 dist-test-slave-vkjp krb5kdc[313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373108, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:08.309731 911 init.cc:377] Logged in from keytab as oryx/127.24.215.189@KRBTEST.COM (short username oryx)
I20260810 14:45:08.311381 911 webserver.cc:460] Webserver started at http://127.24.215.189:43557/ using document root <none> and password file <none>
I20260810 14:45:08.312121 911 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:08.312288 911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:08.312634 911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:08.315837 911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/data/instance:
uuid: "0dab4669ab9e48788241ec344e283a08"
format_stamp: "Formatted at 2026-08-10 14:45:08 on dist-test-slave-vkjp"
server_key: "097da6f82edb317f138e4a5830ac9550"
server_key_iv: "3bba4027de486b37f0f4374dd56f07a2"
server_key_version: "encryptionkey@0"
I20260810 14:45:08.316658 911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/wal/instance:
uuid: "0dab4669ab9e48788241ec344e283a08"
format_stamp: "Formatted at 2026-08-10 14:45:08 on dist-test-slave-vkjp"
server_key: "097da6f82edb317f138e4a5830ac9550"
server_key_iv: "3bba4027de486b37f0f4374dd56f07a2"
server_key_version: "encryptionkey@0"
I20260810 14:45:08.322345 911 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 14:45:08.326156 932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:08.327638 911 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:45:08.327829 911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/wal
uuid: "0dab4669ab9e48788241ec344e283a08"
format_stamp: "Formatted at 2026-08-10 14:45:08 on dist-test-slave-vkjp"
server_key: "097da6f82edb317f138e4a5830ac9550"
server_key_iv: "3bba4027de486b37f0f4374dd56f07a2"
server_key_version: "encryptionkey@0"
I20260810 14:45:08.328101 911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:08.351997 911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:08.352671 745 heartbeater.cc:503] Master 127.24.215.190:41267 was elected leader, sending a full tablet report...
I20260810 14:45:08.357088 911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:08.357383 911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:08.377816 911 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.189:36703
I20260810 14:45:08.377823 995 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.189:36703 every 8 connection(s)
I20260810 14:45:08.379343 911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/data/info.pb
I20260810 14:45:08.383476 996 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:08.385115 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 911
I20260810 14:45:08.393715 996 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:45:08 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.215.189@KRBTEST.COM
Aug 10 14:45:08 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373108, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.189@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM
I20260810 14:45:08.409144 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:08.387552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37982 (local address 127.24.215.189:36703)
0810 14:45:08.388465 (+ 913us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:08.388481 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:08.388541 (+ 60us) server_negotiation.cc:407] Connection header received
0810 14:45:08.390326 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:08.390360 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:08.390908 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:08.391398 (+ 490us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:08.392766 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.394865 (+ 2099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.396254 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.397458 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.397659 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:08.401873 (+ 4214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:08.401881 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:08.401891 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:08.401936 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:08.405155 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:08.405842 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:08.405849 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:08.405855 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:08.406008 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:08.406400 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:08.406412 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:08.406415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:08.406908 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:08.407256 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:08.407731 (+ 475us) server_negotiation.cc:299] Negotiation successful
0810 14:45:08.408128 (+ 397us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":647,"thread_start_us":407,"threads_started":1}
I20260810 14:45:08.411487 1001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:08.396630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33904 (local address 127.24.215.190:41267)
0810 14:45:08.397230 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:08.397239 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:08.397920 (+ 681us) server_negotiation.cc:407] Connection header received
0810 14:45:08.398093 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:08.398105 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:08.398182 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:08.398297 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:08.399509 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.400227 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.401687 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.402104 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.402247 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:08.405587 (+ 3340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:08.405592 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:08.405595 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:08.405627 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:08.408528 (+ 2901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:08.409278 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:08.409286 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:08.409292 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:08.409432 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:08.409848 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:08.409855 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:08.409858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:08.410206 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:08.410366 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:08.410864 (+ 498us) server_negotiation.cc:299] Negotiation successful
0810 14:45:08.411131 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":293,"threads_started":1}
I20260810 14:45:08.411590 1002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:08.396609 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.0.0.1:33904)
0810 14:45:08.397717 (+ 1108us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:08.397775 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:08.397948 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:08.398366 (+ 418us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:08.398373 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:08.399022 (+ 649us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:08.399342 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:08.399352 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.400373 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.400380 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:08.401536 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:08.401548 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.402299 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.402305 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:08.402511 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:08.403146 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:08.403166 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:08.405371 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:08.408659 (+ 3288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:08.408671 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:08.408682 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:08.409142 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:08.409562 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:08.409567 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:08.409571 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:08.409727 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:08.410361 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:08.410369 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:08.410555 (+ 186us) client_negotiation.cc:769] Sending connection context
0810 14:45:08.410768 (+ 213us) client_negotiation.cc:240] Negotiation successful
0810 14:45:08.411130 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":849,"thread_start_us":678,"threads_started":1}
I20260810 14:45:08.414301 996 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:45:08 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373108, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.189@KRBTEST.COM for oryx/127.24.215.189@KRBTEST.COM
I20260810 14:45:08.442648 1002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:08.415829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36703 (local address 127.0.0.1:37992)
0810 14:45:08.416235 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:08.416290 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:08.416561 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:08.417158 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:08.417178 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:08.417889 (+ 711us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:08.418236 (+ 347us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:08.418246 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.420757 (+ 2511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.420774 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:08.423767 (+ 2993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:08.423791 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.425555 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.425569 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:08.425829 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:08.426975 (+ 1146us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:08.427023 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:08.431229 (+ 4206us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:08.437224 (+ 5995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:08.437253 (+ 29us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:08.437271 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:08.438072 (+ 801us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:08.438824 (+ 752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:08.438839 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:08.438846 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:08.439180 (+ 334us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:08.440347 (+ 1167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:08.440371 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:08.440690 (+ 319us) client_negotiation.cc:769] Sending connection context
0810 14:45:08.441041 (+ 351us) client_negotiation.cc:240] Negotiation successful
0810 14:45:08.441722 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102}
I20260810 14:45:08.442754 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:08.416406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37992 (local address 127.24.215.189:36703)
0810 14:45:08.416636 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:08.416649 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:08.416715 (+ 66us) server_negotiation.cc:407] Connection header received
0810 14:45:08.416782 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:08.416795 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:08.416947 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:08.417120 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:08.418444 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.420481 (+ 2037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.424021 (+ 3540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.425287 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.425509 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:08.431657 (+ 6148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:08.431667 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:08.431682 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:08.431770 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:08.436914 (+ 5144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:08.438301 (+ 1387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:08.438331 (+ 30us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:08.438336 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:08.438580 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:08.439497 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:08.439517 (+ 20us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:08.439525 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:08.440086 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:08.440450 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:08.441290 (+ 840us) server_negotiation.cc:299] Negotiation successful
0810 14:45:08.441889 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":236,"server-negotiator.queue_time_us":78}
I20260810 14:45:08.453053 996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: Bootstrap starting.
I20260810 14:45:08.460640 996 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:08.463406 996 log.cc:824] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:08.470505 996 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: No bootstrap required, opened a new log
I20260810 14:45:08.488677 996 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } }
I20260810 14:45:08.489650 996 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:08.489967 996 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0dab4669ab9e48788241ec344e283a08, State: Initialized, Role: FOLLOWER
I20260810 14:45:08.491200 996 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } }
I20260810 14:45:08.493146 1004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } } }
I20260810 14:45:08.493799 1004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:45:08.495558 996 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 14:45:08.517964 1019 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:45:08.518332 1019 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:45:08.523598 911 master_runner.cc:428] Detected that this master 0dab4669ab9e48788241ec344e283a08 is joining an existing cluster
I20260810 14:45:08.523797 911 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.215.189:36703
WARNING: no policy specified for oryx/127.24.215.188@KRBTEST.COM; defaulting to no policy
I20260810 14:45:08.586822 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:08.539779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38000 (local address 127.24.215.189:36703)
0810 14:45:08.540298 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:08.540316 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:08.546808 (+ 6492us) server_negotiation.cc:407] Connection header received
0810 14:45:08.547252 (+ 444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:08.547271 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:08.547429 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:08.547759 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:08.552534 (+ 4775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.554969 (+ 2435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.559264 (+ 4295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.561491 (+ 2227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.561848 (+ 357us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:08.568229 (+ 6381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:08.568249 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:08.568254 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:08.568336 (+ 82us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:08.577595 (+ 9259us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:08.580347 (+ 2752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:08.580352 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:08.580376 (+ 24us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:08.580781 (+ 405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:08.581837 (+ 1056us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:08.581855 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:08.581858 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:08.582793 (+ 935us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:08.583255 (+ 462us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:08.584638 (+ 1383us) server_negotiation.cc:299] Negotiation successful
0810 14:45:08.585561 (+ 923us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":239,"server-negotiator.queue_time_us":160}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.215.188@KRBTEST.COM" created.
I20260810 14:45:08.589157 1001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:08.537184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33918 (local address 127.24.215.190:41267)
0810 14:45:08.537647 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:08.537669 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:08.539197 (+ 1528us) server_negotiation.cc:407] Connection header received
0810 14:45:08.539954 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:08.539973 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:08.540278 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:08.540529 (+ 251us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:08.547398 (+ 6869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.550219 (+ 2821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.556399 (+ 6180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.558276 (+ 1877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.558514 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:08.568391 (+ 9877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:08.568428 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:08.568447 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:08.568548 (+ 101us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:08.578859 (+ 10311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:08.581487 (+ 2628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:08.581495 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:08.581499 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:08.582028 (+ 529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:08.583273 (+ 1245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:08.583283 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:08.583300 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:08.584146 (+ 846us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:08.584890 (+ 744us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:08.587405 (+ 2515us) server_negotiation.cc:299] Negotiation successful
0810 14:45:08.588200 (+ 795us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260810 14:45:08.590770 1026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:08.537218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.0.0.1:33918)
0810 14:45:08.538923 (+ 1705us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:08.538959 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:08.539552 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:08.541187 (+ 1635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:08.541208 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:08.543877 (+ 2669us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:08.545223 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:08.545276 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.550797 (+ 5521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.550819 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:08.555784 (+ 4965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:08.555810 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.559542 (+ 3732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.559592 (+ 50us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:08.562950 (+ 3358us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:08.564375 (+ 1425us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:08.564454 (+ 79us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:08.567856 (+ 3402us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:08.579421 (+ 11565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:08.579438 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:08.579462 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:08.581123 (+ 1661us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:08.582366 (+ 1243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:08.582385 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:08.582389 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:08.582825 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:08.585055 (+ 2230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:08.585062 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:08.585594 (+ 532us) client_negotiation.cc:769] Sending connection context
0810 14:45:08.586251 (+ 657us) client_negotiation.cc:240] Negotiation successful
0810 14:45:08.590074 (+ 3823us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1356,"mutex_wait_us":283,"thread_start_us":991,"threads_started":1}
I20260810 14:45:08.586838 1030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:08.541067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36703 (local address 127.0.0.1:38000)
0810 14:45:08.546508 (+ 5441us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:08.546554 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:08.546930 (+ 376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:08.548286 (+ 1356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:08.548291 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:08.550776 (+ 2485us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:08.552100 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:08.552164 (+ 64us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.555643 (+ 3479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.555650 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:08.558867 (+ 3217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:08.558885 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.561974 (+ 3089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.562003 (+ 29us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:08.562783 (+ 780us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:08.564548 (+ 1765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:08.564596 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:08.567762 (+ 3166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:08.578067 (+ 10305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:08.578081 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:08.578085 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:08.580028 (+ 1943us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:08.581143 (+ 1115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:08.581160 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:08.581164 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:08.581497 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:08.583377 (+ 1880us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:08.583405 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:08.583837 (+ 432us) client_negotiation.cc:769] Sending connection context
0810 14:45:08.584246 (+ 409us) client_negotiation.cc:240] Negotiation successful
0810 14:45:08.585168 (+ 922us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5085,"mutex_wait_us":100,"thread_start_us":4614,"threads_started":1}
I20260810 14:45:08.593888 1004 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:45:08.595072 1004 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } }
I20260810 14:45:08.600817 1004 leader_election.cc:288] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c5cde194e5e348ceac077b3a924912ec (127.24.215.190:41267)
I20260810 14:45:08.601508 376 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0dab4669ab9e48788241ec344e283a08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c5cde194e5e348ceac077b3a924912ec" is_pre_election: true
I20260810 14:45:08.606391 933 leader_election.cc:302] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: 0dab4669ab9e48788241ec344e283a08; no voters: c5cde194e5e348ceac077b3a924912ec
I20260810 14:45:08.609905 1004 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260810 14:45:08.610209 939 tablet.cc:2369] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 10 14:45:08 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373106, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.190@KRBTEST.COM for oryx/127.24.215.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.215.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.188.keytab.
Entry for principal oryx/127.24.215.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.188.keytab.
W20260810 14:45:08.691499 25438 mini_kdc.cc:262] Time spent creating service keytab for oryx/127.24.215.188: real 0.170s user 0.004s sys 0.041s
I20260810 14:45:08.694074 999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:08.618483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38002 (local address 127.24.215.189:36703)
0810 14:45:08.619249 (+ 766us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:08.619307 (+ 58us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:08.620732 (+ 1425us) server_negotiation.cc:407] Connection header received
0810 14:45:08.626195 (+ 5463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:08.626215 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:08.627033 (+ 818us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:08.627781 (+ 748us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:08.636238 (+ 8457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.640347 (+ 4109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.647190 (+ 6843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.651116 (+ 3926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.651631 (+ 515us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:08.668273 (+ 16642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:08.668297 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:08.668345 (+ 48us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:08.668535 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:08.681967 (+ 13432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:08.685494 (+ 3527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:08.685573 (+ 79us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:08.685590 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:08.686163 (+ 573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:08.687440 (+ 1277us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:08.687461 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:08.687487 (+ 26us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:08.688863 (+ 1376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:08.689484 (+ 621us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:08.692313 (+ 2829us) server_negotiation.cc:299] Negotiation successful
0810 14:45:08.692850 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271}
I20260810 14:45:08.694245 1031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:08.617236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36703 (local address 127.0.0.1:38002)
0810 14:45:08.620210 (+ 2974us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:08.620358 (+ 148us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:08.621532 (+ 1174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:08.629446 (+ 7914us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:08.629470 (+ 24us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:08.634015 (+ 4545us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:08.635329 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:08.635374 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.641055 (+ 5681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.641106 (+ 51us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:08.646412 (+ 5306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:08.646487 (+ 75us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:08.652696 (+ 6209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:08.652746 (+ 50us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:08.653988 (+ 1242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:08.656488 (+ 2500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:08.656583 (+ 95us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:08.667258 (+ 10675us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:08.682951 (+ 15693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:08.683007 (+ 56us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:08.683042 (+ 35us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:08.684976 (+ 1934us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:08.686511 (+ 1535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:08.686520 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:08.686526 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:08.686929 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:08.689990 (+ 3061us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:08.690051 (+ 61us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:08.690833 (+ 782us) client_negotiation.cc:769] Sending connection context
0810 14:45:08.691833 (+ 1000us) client_negotiation.cc:240] Negotiation successful
0810 14:45:08.692850 (+ 1017us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1463,"thread_start_us":2116,"threads_started":1}
I20260810 14:45:08.698436 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.188:35031
--webserver_interface=127.24.215.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.188.keytab
--principal=oryx/127.24.215.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:41267,127.24.215.189:36703,127.24.215.188:35031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:45:08.700299 363 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.215.189:36703: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "0dab4669ab9e48788241ec344e283a08"
member_type: NON_VOTER
last_known_addr {
host: "127.24.215.189"
port: 36703
}
attrs {
promote: true
}
}
dest_uuid: "c5cde194e5e348ceac077b3a924912ec"
cas_config_opid_index: -1
I20260810 14:45:08.703464 363 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } }
I20260810 14:45:08.728278 965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:08.732818 1037 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0dab4669ab9e48788241ec344e283a08 (127.24.215.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } } }
I20260810 14:45:08.743932 965 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 1 FOLLOWER]: Refusing update from remote peer c5cde194e5e348ceac077b3a924912ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260810 14:45:08.747635 1036 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.001s
I20260810 14:45:08.782260 1037 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.215.189:36703
I20260810 14:45:08.793519 1036 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } } }
I20260810 14:45:08.794823 1036 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:08.794780 911 master.cc:537] Master@127.24.215.189:36703 shutting down...
I20260810 14:45:08.872896 905 heartbeater.cc:503] Master 127.24.215.190:41267 was elected leader, sending a full tablet report...
I20260810 14:45:08.909082 965 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0dab4669ab9e48788241ec344e283a08 (127.24.215.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } } }
I20260810 14:45:08.928622 1050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } } }
I20260810 14:45:08.929972 1050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: This master's current role is: LEARNER
I20260810 14:45:08.930033 1047 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } } }
I20260810 14:45:08.931365 1047 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: This master's current role is: LEARNER
I20260810 14:45:08.932811 1004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5cde194e5e348ceac077b3a924912ec. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } } }
I20260810 14:45:08.933730 1004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: This master's current role is: LEARNER
I20260810 14:45:08.943956 911 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 1 LEARNER]: Raft consensus shutting down.
I20260810 14:45:08.946578 911 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 1 LEARNER]: Raft consensus is shut down!
I20260810 14:45:08.947108 911 tablet_replica.cc:330] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: stopping tablet replica
I20260810 14:45:08.969594 911 master.cc:552] Master@127.24.215.189:36703 shutdown complete.
I20260810 14:45:08.970000 911 master_runner.cc:305] Clearing existing system tablet
I20260810 14:45:08.971339 911 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:08.971619 911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:08.979061 911 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 14:45:08.983851 1052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:08.984941 911 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:08.986167 911 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:45:08.986567 911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/wal
uuid: "0dab4669ab9e48788241ec344e283a08"
format_stamp: "Formatted at 2026-08-10 14:45:08 on dist-test-slave-vkjp"
server_key: "097da6f82edb317f138e4a5830ac9550"
server_key_iv: "3bba4027de486b37f0f4374dd56f07a2"
server_key_version: "encryptionkey@0"
I20260810 14:45:08.995326 911 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:09.009267 911 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 14:45:09.009828 911 log.cc:1196] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260810 14:45:09.011256 911 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: Deleting consensus metadata
I20260810 14:45:09.012038 911 master_runner.cc:315] Copying system tablet from 127.24.215.190:41267
I20260810 14:45:09.019186 911 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: tablet copy: Beginning tablet copy session from remote peer at address 127.24.215.190:41267
I20260810 14:45:09.055953 1001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:09.021251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33932 (local address 127.24.215.190:41267)
0810 14:45:09.022169 (+ 918us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:09.022192 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:09.024098 (+ 1906us) server_negotiation.cc:407] Connection header received
0810 14:45:09.024750 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:09.024758 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:09.024880 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:09.025164 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:09.028422 (+ 3258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:09.030004 (+ 1582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:09.032669 (+ 2665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:09.034151 (+ 1482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:09.034803 (+ 652us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:09.040957 (+ 6154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:09.040964 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:09.040980 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:09.041101 (+ 121us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:09.048658 (+ 7557us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:09.049981 (+ 1323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:09.050000 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:09.050012 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:09.050239 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:09.051691 (+ 1452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:09.051697 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:09.051699 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:09.052511 (+ 812us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:09.052963 (+ 452us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:09.053969 (+ 1006us) server_negotiation.cc:299] Negotiation successful
0810 14:45:09.054758 (+ 789us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388}
I20260810 14:45:09.056044 1059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:09.020998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.0.0.1:33932)
0810 14:45:09.023703 (+ 2705us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:09.023722 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:09.024404 (+ 682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:09.025349 (+ 945us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:09.025371 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:09.026866 (+ 1495us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:09.027766 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:09.027794 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:09.030300 (+ 2506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:09.030306 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:09.032396 (+ 2090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:09.032422 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:09.035629 (+ 3207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:09.035650 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:09.036020 (+ 370us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:09.037826 (+ 1806us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:09.037885 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:09.040339 (+ 2454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:09.048948 (+ 8609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:09.048956 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:09.048963 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:09.049586 (+ 623us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:09.050727 (+ 1141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:09.050766 (+ 39us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:09.050773 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:09.051446 (+ 673us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:09.052816 (+ 1370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:09.052834 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:09.053350 (+ 516us) client_negotiation.cc:769] Sending connection context
0810 14:45:09.053643 (+ 293us) client_negotiation.cc:240] Negotiation successful
0810 14:45:09.054749 (+ 1106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2281,"thread_start_us":1812,"threads_started":1}
I20260810 14:45:09.059751 391 tablet_copy_service.cc:140] P c5cde194e5e348ceac077b3a924912ec: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 0dab4669ab9e48788241ec344e283a08 ({username='oryx', principal='oryx/127.24.215.189@KRBTEST.COM'} at 127.0.0.1:33932)
I20260810 14:45:09.060302 391 tablet_copy_service.cc:161] P c5cde194e5e348ceac077b3a924912ec: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 0dab4669ab9e48788241ec344e283a08 at {username='oryx', principal='oryx/127.24.215.189@KRBTEST.COM'} at 127.0.0.1:33932: session id = 0dab4669ab9e48788241ec344e283a08-00000000000000000000000000000000
I20260810 14:45:09.069326 391 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 14:45:09.074775 911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 14:45:09.087348 911 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: tablet copy: Starting download of 0 data blocks...
I20260810 14:45:09.088114 911 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: tablet copy: Starting download of 1 WAL segments...
I20260810 14:45:09.094059 911 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 14:45:09.104362 391 tablet_copy_service.cc:342] P c5cde194e5e348ceac077b3a924912ec: Request end of tablet copy session 0dab4669ab9e48788241ec344e283a08-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.215.189@KRBTEST.COM'} at 127.0.0.1:33932
I20260810 14:45:09.104923 391 tablet_copy_service.cc:433] P c5cde194e5e348ceac077b3a924912ec: ending tablet copy session 0dab4669ab9e48788241ec344e283a08-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 0dab4669ab9e48788241ec344e283a08
I20260810 14:45:09.137528 911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:09.143923 1060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:09.146467 1061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:09.155890 1063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:09.158001 911 server_base.cc:1034] running on GCE node
I20260810 14:45:09.159039 911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:09.160930 911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:09.162334 911 hybrid_clock.cc:648] HybridClock initialized: now 1786373109162111 us; error 191 us; skew 500 ppm
I20260810 14:45:09.162760 911 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:09 dist-test-slave-vkjp krb5kdc[313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373109, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:09.167641 911 init.cc:377] Logged in from keytab as oryx/127.24.215.189@KRBTEST.COM (short username oryx)
I20260810 14:45:09.173138 911 webserver.cc:460] Webserver started at http://127.24.215.189:41327/ using document root <none> and password file <none>
I20260810 14:45:09.173899 911 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:09.174095 911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:09.188776 911 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.009s sys 0.001s
I20260810 14:45:09.201577 1073 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:09.202793 911 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.002s sys 0.007s
I20260810 14:45:09.203236 911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/wal
uuid: "0dab4669ab9e48788241ec344e283a08"
format_stamp: "Formatted at 2026-08-10 14:45:08 on dist-test-slave-vkjp"
server_key: "097da6f82edb317f138e4a5830ac9550"
server_key_iv: "3bba4027de486b37f0f4374dd56f07a2"
server_key_version: "encryptionkey@0"
I20260810 14:45:09.203797 911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:09.262434 911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:09.270434 911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:09.343739 1037 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } } }
I20260810 14:45:09.344484 1037 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:09.353428 1127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:09.345719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36703 (local address 127.0.0.1:38018)
0810 14:45:09.352095 (+ 6376us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:09.352641 (+ 546us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5866,"thread_start_us":5450,"threads_started":1}
W20260810 14:45:09.354265 347 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36703: connect: Connection refused (error 111)
I20260810 14:45:09.366500 1127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:09.365069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36703 (local address 127.0.0.1:38028)
0810 14:45:09.365571 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:09.365802 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36703: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223}
W20260810 14:45:09.369746 347 consensus_peers.cc:487] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec -> Peer 0dab4669ab9e48788241ec344e283a08 (127.24.215.189:36703): Couldn't send request to peer 0dab4669ab9e48788241ec344e283a08. Status: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36703: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 14:45:09.381880 911 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.189:36703
I20260810 14:45:09.381986 1141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.189:36703 every 8 connection(s)
I20260810 14:45:09.384120 911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-1/data/info.pb
I20260810 14:45:09.398056 1142 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:45:09.404012 1142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: Bootstrap starting.
I20260810 14:45:09.430719 1142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:45:09.432412 1142 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: Bootstrap complete.
I20260810 14:45:09.435992 1142 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } }
I20260810 14:45:09.437197 1142 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 0dab4669ab9e48788241ec344e283a08, State: Initialized, Role: LEARNER
I20260810 14:45:09.437857 1142 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } }
I20260810 14:45:09.440367 1146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } } }
I20260810 14:45:09.441181 1142 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:45:09.441232 1146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: This master's current role is: LEARNER
I20260810 14:45:09.460223 1158 catalog_manager.cc:1259] Loaded cluster ID: 0602c91b97ca4eb48e7748d24ec2bf1b
I20260810 14:45:09.462121 1158 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: loading cluster ID for follower catalog manager: success
I20260810 14:45:09.469530 1158 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: acquiring CA information for follower catalog manager: success
I20260810 14:45:09.473402 1158 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:45:09.778399 1125 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } } }
I20260810 14:45:09.778787 1125 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:09.788339 1127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:09.778794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36703 (local address 127.0.0.1:38044)
0810 14:45:09.779271 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:09.779292 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:09.779449 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:09.780729 (+ 1280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:09.780737 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:09.780775 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:09.781005 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:09.781032 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:09.783301 (+ 2269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:09.783308 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:09.785392 (+ 2084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:09.785408 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:09.787312 (+ 1904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:09.787328 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:09.787589 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:09.787690 (+ 101us) client_negotiation.cc:769] Sending connection context
0810 14:45:09.787831 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 14:45:09.787982 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342}
I20260810 14:45:09.788710 1159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:09.779489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38044 (local address 127.24.215.189:36703)
0810 14:45:09.780216 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:09.780223 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:09.780259 (+ 36us) server_negotiation.cc:407] Connection header received
0810 14:45:09.780387 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:09.780392 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:09.780508 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:09.780655 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:09.781218 (+ 563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:09.783150 (+ 1932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:09.785604 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:09.787074 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:09.787362 (+ 288us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:09.787560 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:09.787870 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 14:45:09.788032 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":352,"threads_started":1}
I20260810 14:45:09.790175 1146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5cde194e5e348ceac077b3a924912ec. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } } }
I20260810 14:45:09.790670 1146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: This master's current role is: LEARNER
I20260810 14:45:09.791328 1037 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec: attempting to promote NON_VOTER 0dab4669ab9e48788241ec344e283a08 to VOTER
I20260810 14:45:09.792660 1037 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } }
I20260810 14:45:09.796743 1105 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 1 LEARNER]: Refusing update from remote peer c5cde194e5e348ceac077b3a924912ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260810 14:45:09.797495 1037 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260810 14:45:09.802451 1124 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 0dab4669ab9e48788241ec344e283a08 (127.24.215.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } }
I20260810 14:45:09.803406 1105 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 1 FOLLOWER]: Committing config change with OpId 1.6: config changed from index 5 to 6, 0dab4669ab9e48788241ec344e283a08 (127.24.215.189) changed from NON_VOTER to VOTER. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } }
I20260810 14:45:09.807907 1124 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } }
I20260810 14:45:09.808282 1124 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:09.808728 1124 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } }
I20260810 14:45:09.809041 1124 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:09.813403 1146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } }
I20260810 14:45:09.813660 1146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: This master's current role is: FOLLOWER
W20260810 14:45:09.849623 1040 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:09.849931 1040 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:09.850024 1040 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:09.858819 1040 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:45:09.858970 1040 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:09.859051 1040 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:09.859232 1040 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:45:09.859292 1040 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:45:09.871395 1040 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:41267,127.24.215.189:36703,127.24.215.188:35031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.188.keytab
--principal=oryx/127.24.215.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.188:35031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.215.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:09.872713 1040 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:09.873994 1040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:09.891526 1168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:09.892289 1170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:09.891409 1167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:09.893273 1040 server_base.cc:1034] running on GCE node
I20260810 14:45:09.894465 1040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:09.896785 1040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:09.898082 1040 hybrid_clock.cc:648] HybridClock initialized: now 1786373109897976 us; error 72 us; skew 500 ppm
I20260810 14:45:09.898658 1040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:09 dist-test-slave-vkjp krb5kdc[313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373109, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:09.904563 1040 init.cc:377] Logged in from keytab as oryx/127.24.215.188@KRBTEST.COM (short username oryx)
I20260810 14:45:09.906788 1040 webserver.cc:460] Webserver started at http://127.24.215.188:32787/ using document root <none> and password file <none>
I20260810 14:45:09.907693 1040 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:09.907922 1040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:09.908280 1040 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:09.911736 1040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/data/instance:
uuid: "9017f74124d749f0b0a6dc4e4386bf1b"
format_stamp: "Formatted at 2026-08-10 14:45:09 on dist-test-slave-vkjp"
server_key: "1ad56f4cb5684338e8ab464dd3b0f21b"
server_key_iv: "1e1e1a938af0a0805eba11ace3fd9fd3"
server_key_version: "encryptionkey@0"
I20260810 14:45:09.912606 1040 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/wal/instance:
uuid: "9017f74124d749f0b0a6dc4e4386bf1b"
format_stamp: "Formatted at 2026-08-10 14:45:09 on dist-test-slave-vkjp"
server_key: "1ad56f4cb5684338e8ab464dd3b0f21b"
server_key_iv: "1e1e1a938af0a0805eba11ace3fd9fd3"
server_key_version: "encryptionkey@0"
I20260810 14:45:09.918800 1040 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260810 14:45:09.923054 1177 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:09.924585 1040 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:45:09.924765 1040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/wal
uuid: "9017f74124d749f0b0a6dc4e4386bf1b"
format_stamp: "Formatted at 2026-08-10 14:45:09 on dist-test-slave-vkjp"
server_key: "1ad56f4cb5684338e8ab464dd3b0f21b"
server_key_iv: "1e1e1a938af0a0805eba11ace3fd9fd3"
server_key_version: "encryptionkey@0"
I20260810 14:45:09.925024 1040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:09.948930 1040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:09.954033 1040 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:09.954352 1040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:09.984687 1040 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.188:35031
I20260810 14:45:09.984740 1242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.188:35031 every 8 connection(s)
I20260810 14:45:09.986393 1040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/data/info.pb
I20260810 14:45:09.989987 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 1040
I20260810 14:45:09.991648 1243 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:10.001319 1243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:45:09 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.215.188@KRBTEST.COM
Aug 10 14:45:10 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373109, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.188@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM
I20260810 14:45:10.017076 1247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:09.992450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59722 (local address 127.24.215.188:35031)
0810 14:45:09.993268 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:09.993281 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:09.993343 (+ 62us) server_negotiation.cc:407] Connection header received
0810 14:45:09.995302 (+ 1959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:09.995326 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:09.995739 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:09.996163 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:09.997955 (+ 1792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:09.999682 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.001266 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.002016 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.002203 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.008190 (+ 5987us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:10.008201 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:10.008212 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:10.008257 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:10.012066 (+ 3809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.012811 (+ 745us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.012817 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.012823 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.013168 (+ 345us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.013572 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.013577 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.013580 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.014447 (+ 867us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:10.014793 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:10.015197 (+ 404us) server_negotiation.cc:299] Negotiation successful
0810 14:45:10.015578 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":382,"threads_started":1}
I20260810 14:45:10.023463 1249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.004349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.0.0.1:33944)
0810 14:45:10.005884 (+ 1535us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:10.005960 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:10.006167 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:10.006626 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:10.006649 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:10.007642 (+ 993us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:10.008282 (+ 640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.008297 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.009591 (+ 1294us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.009596 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.011150 (+ 1554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.011164 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.012193 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.012201 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.012451 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.013151 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:10.013174 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:10.016222 (+ 3048us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:10.020780 (+ 4558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.020789 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.020794 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.021425 (+ 631us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.021840 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.021843 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.021845 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.021979 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.022564 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:10.022570 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:10.022708 (+ 138us) client_negotiation.cc:769] Sending connection context
0810 14:45:10.022851 (+ 143us) client_negotiation.cc:240] Negotiation successful
0810 14:45:10.023133 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1175,"thread_start_us":817,"threads_started":1}
I20260810 14:45:10.023464 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.004366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33944 (local address 127.24.215.190:41267)
0810 14:45:10.005189 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:10.005196 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:10.006065 (+ 869us) server_negotiation.cc:407] Connection header received
0810 14:45:10.006310 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:10.006314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:10.006430 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:10.006557 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:10.008446 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.009451 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.011291 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.011979 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.012120 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.016477 (+ 4357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:10.016484 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:10.016489 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:10.016532 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:10.020639 (+ 4107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.021543 (+ 904us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.021547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.021550 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.021746 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.022058 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.022061 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.022063 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.022449 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:10.022565 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:10.022954 (+ 389us) server_negotiation.cc:299] Negotiation successful
0810 14:45:10.023179 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":430,"threads_started":1}
I20260810 14:45:10.024817 1243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:45:10 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373109, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.188@KRBTEST.COM for oryx/127.24.215.189@KRBTEST.COM
I20260810 14:45:10.039449 1249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.025503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36703 (local address 127.0.0.1:38050)
0810 14:45:10.025757 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:10.025777 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:10.025864 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:10.026136 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:10.026139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:10.026497 (+ 358us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:10.026636 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.026644 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.027846 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.027851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.029404 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.029419 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.030440 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.030445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.030623 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.031323 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:10.031340 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:10.033597 (+ 2257us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:10.036931 (+ 3334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.036936 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.036940 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.037310 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.037678 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.037683 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.037687 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.037814 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.038349 (+ 535us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:10.038354 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:10.038495 (+ 141us) client_negotiation.cc:769] Sending connection context
0810 14:45:10.038643 (+ 148us) client_negotiation.cc:240] Negotiation successful
0810 14:45:10.038951 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 14:45:10.039534 1159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.025669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38050 (local address 127.24.215.189:36703)
0810 14:45:10.025868 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:10.025873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:10.025891 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:45:10.025962 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:10.025965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:10.026025 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:10.026104 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:10.026761 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.027680 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.029578 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.030261 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.030424 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.033768 (+ 3344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:10.033773 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:10.033776 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:10.033807 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:10.036787 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.037434 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.037440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.037447 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.037565 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.037929 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.037935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.037938 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.038213 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:10.038413 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:10.038738 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 14:45:10.038971 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 14:45:10.040558 1243 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:45:10 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373109, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.188@KRBTEST.COM for oryx/127.24.215.188@KRBTEST.COM
I20260810 14:45:10.055226 1249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.041553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35031 (local address 127.0.0.1:59732)
0810 14:45:10.041771 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:10.041790 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:10.041888 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:10.042201 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:10.042205 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:10.042772 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:10.043037 (+ 265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.043046 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.044080 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.044084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.045067 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.045079 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.045854 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.045858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.046097 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.046729 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:10.046745 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:10.049195 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:10.052510 (+ 3315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.052517 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.052521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.052948 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.053348 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.053354 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.053358 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.053568 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.054082 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:10.054086 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:10.054244 (+ 158us) client_negotiation.cc:769] Sending connection context
0810 14:45:10.054380 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 14:45:10.054656 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 14:45:10.055269 1247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.041597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59732 (local address 127.24.215.188:35031)
0810 14:45:10.041746 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:10.041750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:10.041855 (+ 105us) server_negotiation.cc:407] Connection header received
0810 14:45:10.042017 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:10.042024 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:10.042087 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:10.042162 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:10.043215 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.043965 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.045200 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.045678 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.045778 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.049418 (+ 3640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:10.049423 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:10.049426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:10.049454 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:10.052374 (+ 2920us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.053066 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.053074 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.053078 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.053240 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.053674 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.053682 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.053685 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.053959 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:10.054103 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:10.054472 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 14:45:10.054887 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":118,"server-negotiator.queue_time_us":36}
I20260810 14:45:10.061072 1243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: Bootstrap starting.
I20260810 14:45:10.065169 1243 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:10.066841 1243 log.cc:824] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:10.070084 1243 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: No bootstrap required, opened a new log
I20260810 14:45:10.078132 1243 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } }
I20260810 14:45:10.078559 1243 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:10.078697 1243 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9017f74124d749f0b0a6dc4e4386bf1b, State: Initialized, Role: FOLLOWER
I20260810 14:45:10.079250 1243 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } }
I20260810 14:45:10.080698 1251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } } }
I20260810 14:45:10.081308 1251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:45:10.081352 1243 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: configured and running, proceeding with master startup.
W20260810 14:45:10.093237 1265 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:45:10.093403 1265 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:45:10.098354 1040 master_runner.cc:428] Detected that this master 9017f74124d749f0b0a6dc4e4386bf1b is joining an existing cluster
I20260810 14:45:10.098562 1040 master_runner.cc:432] Initiating AddMaster RPC to add 127.24.215.188:35031
I20260810 14:45:10.123764 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 441
I20260810 14:45:10.128855 1159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.104047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38060 (local address 127.24.215.189:36703)
0810 14:45:10.104482 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:10.104489 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:10.108156 (+ 3667us) server_negotiation.cc:407] Connection header received
0810 14:45:10.108465 (+ 309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:10.108473 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:10.108575 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:10.108677 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:10.110554 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.111988 (+ 1434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.114607 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.115521 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.115656 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.119152 (+ 3496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:10.119177 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:10.119184 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:10.119245 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:10.123755 (+ 4510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.124864 (+ 1109us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.124870 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.124873 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.125076 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.125639 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.125644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.125648 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.126178 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:10.126308 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:10.127992 (+ 1684us) server_negotiation.cc:299] Negotiation successful
0810 14:45:10.128244 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273}
I20260810 14:45:10.130967 1273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.105786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36703 (local address 127.0.0.1:38060)
0810 14:45:10.108057 (+ 2271us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:10.108077 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:10.108291 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:10.108929 (+ 638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:10.108936 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:10.110039 (+ 1103us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:10.110374 (+ 335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.110390 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.112208 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.112217 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.114459 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.114475 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.116020 (+ 1545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.116030 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.116228 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.116914 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:10.116929 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:10.118883 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:10.123944 (+ 5061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.123949 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.123952 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.124682 (+ 730us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.125268 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.125300 (+ 32us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.125304 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.125485 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.126409 (+ 924us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:10.126419 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:10.127577 (+ 1158us) client_negotiation.cc:769] Sending connection context
0810 14:45:10.129141 (+ 1564us) client_negotiation.cc:240] Negotiation successful
0810 14:45:10.129590 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2133,"mutex_wait_us":768,"thread_start_us":1405,"threads_started":1}
I20260810 14:45:10.134440 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.104166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33948 (local address 127.24.215.190:41267)
0810 14:45:10.104824 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:10.104834 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:10.105974 (+ 1140us) server_negotiation.cc:407] Connection header received
0810 14:45:10.106269 (+ 295us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:10.106275 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:10.106391 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:10.106468 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:10.109083 (+ 2615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.110491 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.113420 (+ 2929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.114134 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.114266 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.119166 (+ 4900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:10.119172 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:10.119178 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:10.119220 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:10.123605 (+ 4385us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.124570 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.124575 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.124578 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.124744 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.125262 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.125267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.125270 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.126246 (+ 976us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:10.126488 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:10.133494 (+ 7006us) server_negotiation.cc:299] Negotiation successful
0810 14:45:10.133876 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":489}
I20260810 14:45:10.135998 1272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.104320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.0.0.1:33948)
0810 14:45:10.105816 (+ 1496us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:10.105847 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:10.106064 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:10.106741 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:10.106744 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:10.108121 (+ 1377us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:10.108879 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.108895 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.110733 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.110738 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.113244 (+ 2506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.113266 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.114378 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.114387 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.114952 (+ 565us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.116571 (+ 1619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:10.116599 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:10.118898 (+ 2299us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:10.123769 (+ 4871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.123773 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.123777 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.124429 (+ 652us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.124912 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.124916 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.124919 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.125122 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.126598 (+ 1476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:10.126606 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:10.132912 (+ 6306us) client_negotiation.cc:769] Sending connection context
0810 14:45:10.134648 (+ 1736us) client_negotiation.cc:240] Negotiation successful
0810 14:45:10.135153 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1303,"mutex_wait_us":3547,"thread_start_us":998,"threads_started":1}
I20260810 14:45:10.142236 1247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.104235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59740 (local address 127.24.215.188:35031)
0810 14:45:10.104667 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:10.104674 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:10.115951 (+ 11277us) server_negotiation.cc:407] Connection header received
0810 14:45:10.116179 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:10.116185 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:10.116258 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:10.116370 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:10.119092 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.120364 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.129414 (+ 9050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.130543 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.130675 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.133761 (+ 3086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:10.133766 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:10.133769 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:10.133803 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:10.138306 (+ 4503us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.139448 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.139453 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.139456 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.139603 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.140090 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.140094 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.140097 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.140589 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:10.140838 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:10.141499 (+ 661us) server_negotiation.cc:299] Negotiation successful
0810 14:45:10.141752 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":279}
I20260810 14:45:10.143445 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.107591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35031 (local address 127.0.0.1:59740)
0810 14:45:10.115866 (+ 8275us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:10.115885 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:10.116046 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:10.116414 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:10.116421 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:10.118230 (+ 1809us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:10.118929 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.118943 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.120641 (+ 1698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.120649 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.129180 (+ 8531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.129195 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.131440 (+ 2245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.131446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.131567 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.132144 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:10.132161 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:10.133502 (+ 1341us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:10.138481 (+ 4979us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.138486 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.138490 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.139322 (+ 832us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.139821 (+ 499us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.139825 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.139828 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.139981 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.140940 (+ 959us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:10.140944 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:10.141120 (+ 176us) client_negotiation.cc:769] Sending connection context
0810 14:45:10.142480 (+ 1360us) client_negotiation.cc:240] Negotiation successful
0810 14:45:10.142833 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7618,"thread_start_us":7222,"threads_started":1}
Aug 10 14:45:10 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.215.129@KRBTEST.COM
Aug 10 14:45:10 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373106, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.190@KRBTEST.COM for oryx/127.24.215.188@KRBTEST.COM
I20260810 14:45:10.163509 1275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.125921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48676 (local address 127.24.215.129:46583)
0810 14:45:10.135217 (+ 9296us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:10.135228 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:10.135263 (+ 35us) server_negotiation.cc:407] Connection header received
0810 14:45:10.135340 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:10.135352 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:10.135757 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:10.138263 (+ 2506us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:10.140454 (+ 2191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.143904 (+ 3450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.146884 (+ 2980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.147661 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.147814 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.153504 (+ 5690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:10.153521 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:10.153542 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:10.153601 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:10.158163 (+ 4562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.159308 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.159318 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.159324 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.159566 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.160192 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.160198 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.160202 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.160812 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:10.161211 (+ 399us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:10.162630 (+ 1419us) server_negotiation.cc:299] Negotiation successful
0810 14:45:10.162836 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5022,"thread_start_us":4809,"threads_started":1}
I20260810 14:45:10.165231 585 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:10.165788 1127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.145103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35031 (local address 127.0.0.1:59754)
0810 14:45:10.145485 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:10.145515 (+ 30us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:10.145631 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:10.147223 (+ 1592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:10.147230 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:10.147778 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:10.148116 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.148133 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.150002 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.150008 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.151602 (+ 1594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.151632 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.152858 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.152868 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.153103 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.153835 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:10.153862 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:10.157141 (+ 3279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:10.162396 (+ 5255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.162402 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.162406 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.163128 (+ 722us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.163716 (+ 588us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.163722 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.163726 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.163961 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.164730 (+ 769us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:10.164735 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:10.164889 (+ 154us) client_negotiation.cc:769] Sending connection context
0810 14:45:10.165034 (+ 145us) client_negotiation.cc:240] Negotiation successful
0810 14:45:10.165422 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":243}
I20260810 14:45:10.166325 1247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.146679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59754 (local address 127.24.215.188:35031)
0810 14:45:10.146867 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:10.146872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:10.146889 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:45:10.146972 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:10.146980 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:10.147053 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:10.147171 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:10.148331 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.149808 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.151818 (+ 2010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.152653 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.152804 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.157379 (+ 4575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:10.157387 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:10.157392 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:10.157435 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:10.162200 (+ 4765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.163252 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.163258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.163261 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.163437 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.164106 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.164110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.164114 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.164616 (+ 502us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:10.164760 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:10.165139 (+ 379us) server_negotiation.cc:299] Negotiation successful
0810 14:45:10.165437 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 14:45:10.167855 363 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.24.215.188:35031: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b"
member_type: NON_VOTER
last_known_addr {
host: "127.24.215.188"
port: 35031
}
attrs {
promote: true
}
}
dest_uuid: "c5cde194e5e348ceac077b3a924912ec"
cas_config_opid_index: 6
I20260810 14:45:10.168787 363 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: NON_VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: true } }
I20260810 14:45:10.171497 1105 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 1 FOLLOWER]: Refusing update from remote peer c5cde194e5e348ceac077b3a924912ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 14:45:10.172026 1214 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:10.172206 1125 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260810 14:45:10.175784 1105 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 9017f74124d749f0b0a6dc4e4386bf1b (127.24.215.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: NON_VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: true } } }
I20260810 14:45:10.177290 1214 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 1 FOLLOWER]: Refusing update from remote peer c5cde194e5e348ceac077b3a924912ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260810 14:45:10.178339 1124 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: NON_VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:10.174937 1125 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, NON_VOTER 9017f74124d749f0b0a6dc4e4386bf1b (127.24.215.188) added. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: NON_VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: true } } }
I20260810 14:45:10.195217 1146 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: NON_VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: true } } }
I20260810 14:45:10.195708 1146 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:45:10.196226 1125 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.24.215.188:35031
I20260810 14:45:10.196971 1040 master.cc:537] Master@127.24.215.188:35031 shutting down...
I20260810 14:45:10.198140 1124 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: NON_VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: true } } }
I20260810 14:45:10.198423 1124 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:10.198746 1124 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: NON_VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: true } } }
I20260810 14:45:10.199035 1124 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: This master's current role is: LEADER
W20260810 14:45:10.207002 1184 tablet.cc:2369] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:45:10.211934 1214 raft_consensus.cc:858] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } }
W20260810 14:45:10.212118 1214 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 1 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } }
I20260810 14:45:10.248533 1214 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 0dab4669ab9e48788241ec344e283a08 (127.24.215.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } } }
I20260810 14:45:10.254830 1284 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } } }
I20260810 14:45:10.255268 1284 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 14:45:10.255505 1251 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5cde194e5e348ceac077b3a924912ec. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } } }
I20260810 14:45:10.255718 1251 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 14:45:10.255852 1281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: NON_VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: true } } }
I20260810 14:45:10.256104 1281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260810 14:45:10.256191 1125 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: NON_VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: true } } }
I20260810 14:45:10.256628 1125 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:10.259176 1040 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260810 14:45:10.259660 1040 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
W20260810 14:45:10.259627 346 consensus_peers.cc:487] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec -> Peer 9017f74124d749f0b0a6dc4e4386bf1b (127.24.215.188:35031): Couldn't send request to peer 9017f74124d749f0b0a6dc4e4386bf1b. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260810 14:45:10.259832 1040 tablet_replica.cc:330] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: stopping tablet replica
I20260810 14:45:10.269388 1040 master.cc:552] Master@127.24.215.188:35031 shutdown complete.
I20260810 14:45:10.269598 1040 master_runner.cc:305] Clearing existing system tablet
I20260810 14:45:10.270088 1040 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:10.270290 1040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:10.273365 1040 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.001s
I20260810 14:45:10.275193 1287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:10.275602 1040 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:10.276083 1040 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.001s
I20260810 14:45:10.276297 1040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/wal
uuid: "9017f74124d749f0b0a6dc4e4386bf1b"
format_stamp: "Formatted at 2026-08-10 14:45:09 on dist-test-slave-vkjp"
server_key: "1ad56f4cb5684338e8ab464dd3b0f21b"
server_key_iv: "1e1e1a938af0a0805eba11ace3fd9fd3"
server_key_version: "encryptionkey@0"
I20260810 14:45:10.281970 1040 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:10.289810 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 441
I20260810 14:45:10.296478 1040 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260810 14:45:10.296680 1040 log.cc:1196] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260810 14:45:10.297248 1040 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: Deleting consensus metadata
I20260810 14:45:10.297536 1040 master_runner.cc:315] Copying system tablet from 127.24.215.190:41267
I20260810 14:45:10.304018 1040 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: tablet copy: Beginning tablet copy session from remote peer at address 127.24.215.190:41267
I20260810 14:45:10.314810 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:46583
--local_ip_for_outbound_sockets=127.24.215.129
--tserver_master_addrs=127.24.215.190:41267
--webserver_port=45483
--webserver_interface=127.24.215.129
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.129.keytab
--principal=oryx/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.215.190:41267,127.24.215.189:36703
--tserver_master_addrs=127.24.215.190:41267,127.24.215.189:36703,127.24.215.188:35031 with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:45:10.321769 1293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.304880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.0.0.1:33962)
0810 14:45:10.305415 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:10.305436 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:10.305661 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:10.306141 (+ 480us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:10.306147 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:10.306734 (+ 587us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:10.307055 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.307070 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.308794 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.308805 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.310398 (+ 1593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.310414 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.311568 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.311574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.311755 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.312567 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:10.312593 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:10.314047 (+ 1454us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:10.318468 (+ 4421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.318474 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.318478 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.319004 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.319551 (+ 547us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.319556 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.319560 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.319756 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.320487 (+ 731us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:10.320493 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:10.320713 (+ 220us) client_negotiation.cc:769] Sending connection context
0810 14:45:10.320957 (+ 244us) client_negotiation.cc:240] Negotiation successful
0810 14:45:10.321299 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":608,"threads_started":1}
I20260810 14:45:10.322901 1248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.304909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33962 (local address 127.24.215.190:41267)
0810 14:45:10.305148 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:10.305157 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:10.305533 (+ 376us) server_negotiation.cc:407] Connection header received
0810 14:45:10.305875 (+ 342us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:10.305884 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:10.305984 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:10.306099 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:10.307267 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.308636 (+ 1369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.310559 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.311407 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.311563 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.314256 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:10.314264 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:10.314270 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:10.314315 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:10.318320 (+ 4005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.319220 (+ 900us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.319227 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.319231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.319387 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.319908 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.319914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.319918 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.320276 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:10.320412 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:10.322299 (+ 1887us) server_negotiation.cc:299] Negotiation successful
0810 14:45:10.322541 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 14:45:10.323853 391 tablet_copy_service.cc:140] P c5cde194e5e348ceac077b3a924912ec: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 9017f74124d749f0b0a6dc4e4386bf1b ({username='oryx', principal='oryx/127.24.215.188@KRBTEST.COM'} at 127.0.0.1:33962)
I20260810 14:45:10.324041 391 tablet_copy_service.cc:161] P c5cde194e5e348ceac077b3a924912ec: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 9017f74124d749f0b0a6dc4e4386bf1b at {username='oryx', principal='oryx/127.24.215.188@KRBTEST.COM'} at 127.0.0.1:33962: session id = 9017f74124d749f0b0a6dc4e4386bf1b-00000000000000000000000000000000
I20260810 14:45:10.327742 391 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec: Tablet Copy: opened 0 blocks and 1 log segments
I20260810 14:45:10.329746 1040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260810 14:45:10.336790 1040 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: tablet copy: Starting download of 0 data blocks...
I20260810 14:45:10.337138 1040 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: tablet copy: Starting download of 1 WAL segments...
I20260810 14:45:10.339825 1040 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260810 14:45:10.344310 391 tablet_copy_service.cc:342] P c5cde194e5e348ceac077b3a924912ec: Request end of tablet copy session 9017f74124d749f0b0a6dc4e4386bf1b-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.24.215.188@KRBTEST.COM'} at 127.0.0.1:33962
I20260810 14:45:10.344457 391 tablet_copy_service.cc:433] P c5cde194e5e348ceac077b3a924912ec: ending tablet copy session 9017f74124d749f0b0a6dc4e4386bf1b-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 9017f74124d749f0b0a6dc4e4386bf1b
I20260810 14:45:10.357609 1040 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:10.360442 1295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:10.361759 1296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:10.363436 1298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:10.363833 1040 server_base.cc:1034] running on GCE node
I20260810 14:45:10.364156 1040 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:10.364828 1040 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:10.366063 1040 hybrid_clock.cc:648] HybridClock initialized: now 1786373110366010 us; error 37 us; skew 500 ppm
I20260810 14:45:10.366354 1040 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:10 dist-test-slave-vkjp krb5kdc[313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373110, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:10.368567 1040 init.cc:377] Logged in from keytab as oryx/127.24.215.188@KRBTEST.COM (short username oryx)
I20260810 14:45:10.374924 1040 webserver.cc:460] Webserver started at http://127.24.215.188:35427/ using document root <none> and password file <none>
I20260810 14:45:10.375277 1040 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:10.375407 1040 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:10.377930 1040 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.002s
I20260810 14:45:10.379667 1311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:10.380074 1040 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260810 14:45:10.380196 1040 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/wal
uuid: "9017f74124d749f0b0a6dc4e4386bf1b"
format_stamp: "Formatted at 2026-08-10 14:45:09 on dist-test-slave-vkjp"
server_key: "1ad56f4cb5684338e8ab464dd3b0f21b"
server_key_iv: "1e1e1a938af0a0805eba11ace3fd9fd3"
server_key_version: "encryptionkey@0"
I20260810 14:45:10.380403 1040 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:10.402339 1040 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:10.406620 1040 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:10.439555 1040 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.188:35031
I20260810 14:45:10.439620 1372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.188:35031 every 8 connection(s)
I20260810 14:45:10.440649 1040 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/master-2/data/info.pb
I20260810 14:45:10.446624 1373 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:45:10.450543 1373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: Bootstrap starting.
I20260810 14:45:10.466612 1373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:45:10.467185 1373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: Bootstrap complete.
I20260810 14:45:10.468313 1373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: NON_VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: true } }
I20260810 14:45:10.468640 1373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 9017f74124d749f0b0a6dc4e4386bf1b, State: Initialized, Role: LEARNER
I20260810 14:45:10.469099 1373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: NON_VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: true } }
I20260810 14:45:10.470045 1377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: NON_VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: true } } }
I20260810 14:45:10.470376 1373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:45:10.470542 1377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: This master's current role is: LEARNER
I20260810 14:45:10.480809 1388 catalog_manager.cc:1259] Loaded cluster ID: 0602c91b97ca4eb48e7748d24ec2bf1b
I20260810 14:45:10.480933 1388 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: loading cluster ID for follower catalog manager: success
I20260810 14:45:10.484562 1388 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: acquiring CA information for follower catalog manager: success
I20260810 14:45:10.487001 1388 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 14:45:10.718116 1294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:10.718441 1294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:10.718602 1294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:10.726548 1294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:10.726704 1294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:10.726927 1294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:45:10.737704 1294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.129.keytab
--principal=oryx/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:46583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=45483
--tserver_master_addrs=127.24.215.190:41267,127.24.215.189:36703,127.24.215.188:35031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:10.739259 1294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:10.740656 1294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:10.755465 1395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:10.755491 1398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:10.755764 1396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:10.756968 1294 server_base.cc:1034] running on GCE node
I20260810 14:45:10.757685 1294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:10.758725 1294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:10.759969 1294 hybrid_clock.cc:648] HybridClock initialized: now 1786373110759908 us; error 52 us; skew 500 ppm
I20260810 14:45:10.760382 1294 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:10 dist-test-slave-vkjp krb5kdc[313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373110, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:10.764350 1294 init.cc:377] Logged in from keytab as oryx/127.24.215.129@KRBTEST.COM (short username oryx)
I20260810 14:45:10.765929 1294 webserver.cc:460] Webserver started at http://127.24.215.129:45483/ using document root <none> and password file <none>
I20260810 14:45:10.766757 1294 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:10.766943 1294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:10.773162 1294 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 14:45:10.776441 1405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:10.778189 1294 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:45:10.778347 1294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "770eff9431a14800b13cbb540c4483a6"
format_stamp: "Formatted at 2026-08-10 14:45:06 on dist-test-slave-vkjp"
server_key: "aaf54fa57eddb5aa0a65e78ee50c120f"
server_key_iv: "54006ca97de146ebd715aed66bf08ecd"
server_key_version: "encryptionkey@0"
I20260810 14:45:10.779450 1294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:10.781085 1125 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: NON_VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: true } } }
I20260810 14:45:10.781370 1125 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:10.788604 1410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.782046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59266 (local address 127.24.215.188:35031)
0810 14:45:10.782880 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:10.782885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:10.782902 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:45:10.782973 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:10.782977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:10.783046 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:10.783216 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:10.783691 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.784801 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.786457 (+ 1656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.787519 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.787860 (+ 341us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.788063 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:10.788120 (+ 57us) server_negotiation.cc:299] Negotiation successful
0810 14:45:10.788249 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":707,"thread_start_us":492,"threads_started":1}
I20260810 14:45:10.788666 1409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.781916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35031 (local address 127.0.0.1:59266)
0810 14:45:10.782595 (+ 679us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:10.782616 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:10.782760 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:10.783247 (+ 487us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:10.783254 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:10.783289 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:10.783544 (+ 255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.783555 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.784942 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.784948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.786313 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.786327 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.787736 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.787743 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.787958 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.787997 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 14:45:10.788087 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 14:45:10.788226 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":329,"threads_started":1}
I20260810 14:45:10.789619 1342 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 1 LEARNER]: Deduplicated request from leader. Original: 1.5->[1.6-1.7] Dedup: 1.7->[]
I20260810 14:45:10.790031 1377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader c5cde194e5e348ceac077b3a924912ec. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: NON_VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: true } } }
I20260810 14:45:10.790333 1377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: This master's current role is: LEARNER
I20260810 14:45:10.790683 1407 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec: attempting to promote NON_VOTER 9017f74124d749f0b0a6dc4e4386bf1b to VOTER
I20260810 14:45:10.791527 1407 consensus_queue.cc:237] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } }
I20260810 14:45:10.793463 1105 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 1 FOLLOWER]: Refusing update from remote peer c5cde194e5e348ceac077b3a924912ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 14:45:10.793529 1342 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 1 LEARNER]: Refusing update from remote peer c5cde194e5e348ceac077b3a924912ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260810 14:45:10.794060 1125 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 14:45:10.794334 1406 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260810 14:45:10.797200 1407 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 9017f74124d749f0b0a6dc4e4386bf1b (127.24.215.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } } }
I20260810 14:45:10.797904 1105 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 9017f74124d749f0b0a6dc4e4386bf1b (127.24.215.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } } }
I20260810 14:45:10.800664 1342 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 9017f74124d749f0b0a6dc4e4386bf1b (127.24.215.188) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } } }
I20260810 14:45:10.802274 1406 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } } }
I20260810 14:45:10.802650 1406 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:10.802932 1406 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } } }
I20260810 14:45:10.803256 1406 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:10.803498 1406 sys_catalog.cc:455] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } } }
I20260810 14:45:10.803819 1406 sys_catalog.cc:458] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:10.806439 1377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } } }
I20260810 14:45:10.806749 1377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:45:10.809089 1416 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c5cde194e5e348ceac077b3a924912ec" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } } }
I20260810 14:45:10.809451 1416 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:45:10.812320 1294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:10.817216 1294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:10.817524 1294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:10.818819 1294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:10.820514 1294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:10.820632 1294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:10.820765 1294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:10.820863 1294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:10.858425 1294 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:46583
I20260810 14:45:10.858516 1545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:46583 every 8 connection(s)
I20260810 14:45:10.859966 1294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:45:10.866837 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 1294
I20260810 14:45:10.867211 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 598
Aug 10 14:45:10 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373110, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.189@KRBTEST.COM
Aug 10 14:45:10 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373110, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.188@KRBTEST.COM
Aug 10 14:45:10 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373110, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM
I20260810 14:45:10.893607 1550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.863442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:40313 (local address 127.24.215.189:36703)
0810 14:45:10.864274 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:10.864282 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:10.866014 (+ 1732us) server_negotiation.cc:407] Connection header received
0810 14:45:10.867929 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:10.867935 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:10.868020 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:10.868110 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:10.871837 (+ 3727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.873454 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.877061 (+ 3607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.878000 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.878133 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.885384 (+ 7251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:10.885391 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:10.885396 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:10.885440 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:10.889669 (+ 4229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.890659 (+ 990us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.890665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.890671 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.890830 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.891407 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.891413 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.891417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.891758 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:10.891906 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:10.892941 (+ 1035us) server_negotiation.cc:299] Negotiation successful
0810 14:45:10.893162 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":513,"threads_started":1}
I20260810 14:45:10.893668 1410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.863169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:53599 (local address 127.24.215.188:35031)
0810 14:45:10.863775 (+ 606us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:10.863781 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:10.866191 (+ 2410us) server_negotiation.cc:407] Connection header received
0810 14:45:10.868415 (+ 2224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:10.868424 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:10.868531 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:10.868626 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:10.871821 (+ 3195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.873316 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.878710 (+ 5394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.879731 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.879865 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.886312 (+ 6447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:10.886319 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:10.886324 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:10.886370 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:10.890165 (+ 3795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.891170 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.891177 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.891182 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.891381 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.891862 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.891868 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.891871 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.892227 (+ 356us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:10.892408 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:10.892999 (+ 591us) server_negotiation.cc:299] Negotiation successful
0810 14:45:10.893238 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451}
I20260810 14:45:10.896008 1553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.863743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35031 (local address 127.24.215.129:53599)
0810 14:45:10.866075 (+ 2332us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:10.866096 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:10.867589 (+ 1493us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:10.868902 (+ 1313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:10.868910 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:10.870176 (+ 1266us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:10.871585 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.871606 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.873626 (+ 2020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.873634 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.878532 (+ 4898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.878555 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.880205 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.880212 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.880567 (+ 355us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.881299 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:10.881346 (+ 47us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:10.885418 (+ 4072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:10.890389 (+ 4971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.890395 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.890399 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.890999 (+ 600us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.891552 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.891556 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.891560 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.891722 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.892504 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:10.892509 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:10.892675 (+ 166us) client_negotiation.cc:769] Sending connection context
0810 14:45:10.894021 (+ 1346us) client_negotiation.cc:240] Negotiation successful
0810 14:45:10.894510 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1697,"mutex_wait_us":616,"spinlock_wait_cycles":20352,"thread_start_us":1017,"threads_started":1}
I20260810 14:45:10.896733 1552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.864047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:41357 (local address 127.24.215.190:41267)
0810 14:45:10.864875 (+ 828us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:10.864884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:10.865894 (+ 1010us) server_negotiation.cc:407] Connection header received
0810 14:45:10.867919 (+ 2025us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:10.867924 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:10.868015 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:10.868123 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:10.875224 (+ 7101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.877267 (+ 2043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.882865 (+ 5598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.883648 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.883782 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.888844 (+ 5062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:10.888853 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:10.888858 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:10.888897 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:10.892582 (+ 3685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.893376 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.893381 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.893385 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.893526 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.894072 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.894077 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.894080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.894435 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:10.894579 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:10.896104 (+ 1525us) server_negotiation.cc:299] Negotiation successful
0810 14:45:10.896337 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":464,"threads_started":1}
I20260810 14:45:10.897027 1551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.863743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.129:41357)
0810 14:45:10.865310 (+ 1567us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:10.865374 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:10.867589 (+ 2215us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:10.874209 (+ 6620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:10.874216 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:10.874732 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:10.875048 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.875060 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.877415 (+ 2355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.877421 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.882709 (+ 5288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.882727 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.883825 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.883829 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.884036 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.885755 (+ 1719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:10.885777 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:10.888511 (+ 2734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:10.892737 (+ 4226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.892743 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.892746 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.893229 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.893683 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.893691 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.893697 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.893923 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.894638 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:10.894642 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:10.894784 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 14:45:10.894968 (+ 184us) client_negotiation.cc:240] Negotiation successful
0810 14:45:10.895273 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":609,"mutex_wait_us":3515,"spinlock_wait_cycles":32128,"thread_start_us":338,"threads_started":1}
I20260810 14:45:10.897048 1554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.863743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36703 (local address 127.24.215.129:40313)
0810 14:45:10.865894 (+ 2151us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:10.865918 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:10.867600 (+ 1682us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:10.868712 (+ 1112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:10.868727 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:10.870173 (+ 1446us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:10.871570 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.871604 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.873709 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.873717 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.876840 (+ 3123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:10.876863 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.878640 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.878650 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:10.879337 (+ 687us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.880573 (+ 1236us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:10.880617 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:10.885088 (+ 4471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:10.889839 (+ 4751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.889848 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.889864 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.890509 (+ 645us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.891009 (+ 500us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:10.891014 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:10.891018 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:10.891272 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:10.892001 (+ 729us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:10.892011 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:10.892460 (+ 449us) client_negotiation.cc:769] Sending connection context
0810 14:45:10.893872 (+ 1412us) client_negotiation.cc:240] Negotiation successful
0810 14:45:10.894676 (+ 804us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1988,"mutex_wait_us":1145,"thread_start_us":1195,"threads_started":1}
I20260810 14:45:10.898088 1546 heartbeater.cc:347] Connected to a master server at 127.24.215.188:35031
I20260810 14:45:10.898428 1546 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:10.899224 1548 heartbeater.cc:347] Connected to a master server at 127.24.215.189:36703
Aug 10 14:45:10 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.215.130@KRBTEST.COM
I20260810 14:45:10.899382 1546 heartbeater.cc:511] Master 127.24.215.188:35031 requested a full tablet report, sending...
I20260810 14:45:10.899392 1548 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:10.899575 1547 heartbeater.cc:347] Connected to a master server at 127.24.215.190:41267
I20260810 14:45:10.899770 1547 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:10.900367 1548 heartbeater.cc:511] Master 127.24.215.189:36703 requested a full tablet report, sending...
I20260810 14:45:10.900391 1547 heartbeater.cc:511] Master 127.24.215.190:41267 requested a full tablet report, sending...
I20260810 14:45:10.902318 1330 ts_manager.cc:194] Registered new tserver with Master: 770eff9431a14800b13cbb540c4483a6 (127.24.215.129:46583)
I20260810 14:45:10.902458 1091 ts_manager.cc:194] Registered new tserver with Master: 770eff9431a14800b13cbb540c4483a6 (127.24.215.129:46583)
I20260810 14:45:10.902702 363 ts_manager.cc:194] Re-registered known tserver with Master: 770eff9431a14800b13cbb540c4483a6 (127.24.215.129:46583)
I20260810 14:45:10.903939 363 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:41357
I20260810 14:45:10.908115 1556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:10.868301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33154 (local address 127.24.215.130:35709)
0810 14:45:10.882338 (+ 14037us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:10.882348 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:10.884610 (+ 2262us) server_negotiation.cc:407] Connection header received
0810 14:45:10.884971 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:10.884983 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:10.885274 (+ 291us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:10.885414 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:10.888711 (+ 3297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.890536 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.892470 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:10.893327 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:10.893441 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:10.900812 (+ 7371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:10.900826 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:10.900866 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:10.900912 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:10.905012 (+ 4100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.906011 (+ 999us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.906020 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.906024 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.906200 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:10.906603 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:10.906607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:10.906609 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:10.906942 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:10.907242 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:10.907507 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 14:45:10.907715 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13828,"thread_start_us":14040,"threads_started":1}
I20260810 14:45:10.909426 738 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:10.995863 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 598
I20260810 14:45:11.017712 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:35709
--local_ip_for_outbound_sockets=127.24.215.130
--tserver_master_addrs=127.24.215.190:41267
--webserver_port=43075
--webserver_interface=127.24.215.130
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.130.keytab
--principal=oryx/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.215.190:41267,127.24.215.189:36703
--tserver_master_addrs=127.24.215.190:41267,127.24.215.189:36703,127.24.215.188:35031 with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:11.360855 1558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:11.361169 1558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:11.361326 1558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:11.368680 1558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:11.368837 1558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:11.369004 1558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:45:11.379535 1558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.130.keytab
--principal=oryx/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:35709
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=43075
--tserver_master_addrs=127.24.215.190:41267,127.24.215.189:36703,127.24.215.188:35031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:11.381007 1558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:11.382515 1558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:11.396005 1564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:11.395946 1568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:11.396179 1565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:11.397396 1558 server_base.cc:1034] running on GCE node
I20260810 14:45:11.398033 1558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:11.398979 1558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:11.400206 1558 hybrid_clock.cc:648] HybridClock initialized: now 1786373111400153 us; error 41 us; skew 500 ppm
I20260810 14:45:11.400604 1558 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:11 dist-test-slave-vkjp krb5kdc[313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373111, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:11.403720 1558 init.cc:377] Logged in from keytab as oryx/127.24.215.130@KRBTEST.COM (short username oryx)
I20260810 14:45:11.405061 1558 webserver.cc:460] Webserver started at http://127.24.215.130:43075/ using document root <none> and password file <none>
I20260810 14:45:11.405818 1558 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:11.405961 1558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:11.411304 1558 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 14:45:11.413971 1575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:11.415424 1558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:45:11.415594 1558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "05be3da9c7e74b218e6025789e323cb9"
format_stamp: "Formatted at 2026-08-10 14:45:07 on dist-test-slave-vkjp"
server_key: "5ef73ddcb6694420740d70fee353f658"
server_key_iv: "af4f8e3714250ec22e0ea1c172bf3b3e"
server_key_version: "encryptionkey@0"
I20260810 14:45:11.416565 1558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:11.430361 1558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:11.434239 1558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:11.434531 1558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:11.435477 1558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:11.436771 1558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:11.436887 1558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:11.437008 1558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:11.437093 1558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:11.475973 1558 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:35709
I20260810 14:45:11.476033 1710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:35709 every 8 connection(s)
I20260810 14:45:11.477506 1558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:45:11.478030 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 1558
I20260810 14:45:11.478353 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 750
Aug 10 14:45:11 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373111, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM
Aug 10 14:45:11 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.215.131@KRBTEST.COM
Aug 10 14:45:11 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373111, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.189@KRBTEST.COM
Aug 10 14:45:11 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373111, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.188@KRBTEST.COM
I20260810 14:45:11.524432 1716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:11.480459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54372 (local address 127.24.215.131:34477)
0810 14:45:11.499814 (+ 19355us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:11.499824 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:11.499853 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:45:11.499924 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:11.499932 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:11.500185 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:11.500372 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:11.502870 (+ 2498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:11.504226 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:11.506062 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:11.506890 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:11.507024 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:11.514714 (+ 7690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:11.514731 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:11.514750 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:11.514797 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:11.519299 (+ 4502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:11.520264 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:11.520277 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:11.520281 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:11.520467 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:11.521072 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:11.521079 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:11.521082 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:11.521602 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:11.523299 (+ 1697us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:11.523538 (+ 239us) server_negotiation.cc:299] Negotiation successful
0810 14:45:11.523867 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19153,"thread_start_us":19479,"threads_started":1}
I20260810 14:45:11.524818 1717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:11.481543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.130:49527)
0810 14:45:11.484100 (+ 2557us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:11.484147 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:11.488706 (+ 4559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:11.501622 (+ 12916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:11.501627 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:11.502202 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:11.503147 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:11.503194 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:11.506177 (+ 2983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:11.506182 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:11.507817 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:11.507829 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:11.509003 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:11.509008 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:11.509456 (+ 448us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:11.510297 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:11.510324 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:11.513872 (+ 3548us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:11.519250 (+ 5378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:11.519263 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:11.519285 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:11.520040 (+ 755us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:11.520585 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:11.520590 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:11.520593 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:11.520913 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:11.521736 (+ 823us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:11.521746 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:11.522278 (+ 532us) client_negotiation.cc:769] Sending connection context
0810 14:45:11.522611 (+ 333us) client_negotiation.cc:240] Negotiation successful
0810 14:45:11.523247 (+ 636us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2239,"mutex_wait_us":229,"thread_start_us":1990,"threads_started":1}
I20260810 14:45:11.525504 1723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:11.489924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:48717 (local address 127.24.215.188:35031)
0810 14:45:11.498781 (+ 8857us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:11.498791 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:11.501149 (+ 2358us) server_negotiation.cc:407] Connection header received
0810 14:45:11.501223 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:11.501228 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:11.501322 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:11.501435 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:11.503398 (+ 1963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:11.504606 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:11.507733 (+ 3127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:11.508468 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:11.508595 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:11.516045 (+ 7450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:11.516052 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:11.516056 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:11.516100 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:11.520446 (+ 4346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:11.521523 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:11.521529 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:11.521533 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:11.521687 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:11.522185 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:11.522190 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:11.522194 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:11.522571 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:11.522709 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:11.524838 (+ 2129us) server_negotiation.cc:299] Negotiation successful
0810 14:45:11.525121 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8684,"thread_start_us":8587,"threads_started":1}
I20260810 14:45:11.525704 1719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:11.488319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:57715 (local address 127.24.215.189:36703)
0810 14:45:11.499520 (+ 11201us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:11.499526 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:11.499548 (+ 22us) server_negotiation.cc:407] Connection header received
0810 14:45:11.499618 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:11.499622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:11.499702 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:11.499823 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:11.503397 (+ 3574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:11.504606 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:11.508098 (+ 3492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:11.508813 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:11.508939 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:11.515968 (+ 7029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:11.515979 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:11.515984 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:11.516029 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:11.520565 (+ 4536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:11.521868 (+ 1303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:11.521873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:11.521877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:11.522023 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:11.522489 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:11.522494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:11.522497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:11.522864 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:11.522999 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:11.524559 (+ 1560us) server_negotiation.cc:299] Negotiation successful
0810 14:45:11.524844 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11065,"thread_start_us":10882,"threads_started":1}
I20260810 14:45:11.525943 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:11.489998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35031 (local address 127.24.215.130:48717)
0810 14:45:11.500624 (+ 10626us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:11.500642 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:11.500765 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:11.501631 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:11.501635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:11.502360 (+ 725us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:11.503147 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:11.503193 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:11.504801 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:11.504817 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:11.507533 (+ 2716us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:11.507551 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:11.508952 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:11.508963 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:11.509448 (+ 485us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:11.510292 (+ 844us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:11.510374 (+ 82us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:11.515765 (+ 5391us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:11.520615 (+ 4850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:11.520619 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:11.520622 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:11.521366 (+ 744us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:11.521824 (+ 458us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:11.521829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:11.521834 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:11.522045 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:11.522779 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:11.522784 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:11.522969 (+ 185us) client_negotiation.cc:769] Sending connection context
0810 14:45:11.523166 (+ 197us) client_negotiation.cc:240] Negotiation successful
0810 14:45:11.523474 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10493,"mutex_wait_us":323,"spinlock_wait_cycles":20352,"thread_start_us":10300,"threads_started":1}
I20260810 14:45:11.526221 1718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:11.482137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:49527 (local address 127.24.215.190:41267)
0810 14:45:11.501287 (+ 19150us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:11.501293 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:11.501322 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:45:11.501390 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:11.501396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:11.501478 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:11.501602 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:11.504809 (+ 3207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:11.505945 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:11.507968 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:11.508645 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:11.508764 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:11.514129 (+ 5365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:11.514135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:11.514138 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:11.514192 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:11.518297 (+ 4105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:11.520233 (+ 1936us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:11.520239 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:11.520242 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:11.520432 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:11.521101 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:11.521106 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:11.521113 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:11.521505 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:11.521661 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:11.525383 (+ 3722us) server_negotiation.cc:299] Negotiation successful
0810 14:45:11.525645 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18993,"thread_start_us":18851,"threads_started":1}
I20260810 14:45:11.526238 1721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:11.488954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36703 (local address 127.24.215.130:57715)
0810 14:45:11.498120 (+ 9166us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:11.498143 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:11.498279 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:11.500149 (+ 1870us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:11.500164 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:11.501343 (+ 1179us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:11.503227 (+ 1884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:11.503238 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:11.504799 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:11.504806 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:11.507535 (+ 2729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:11.507550 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:11.509250 (+ 1700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:11.509258 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:11.509530 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:11.510501 (+ 971us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:11.510535 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:11.515711 (+ 5176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:11.520887 (+ 5176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:11.520898 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:11.520903 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:11.521628 (+ 725us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:11.522163 (+ 535us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:11.522168 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:11.522171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:11.522344 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:11.523062 (+ 718us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:11.523067 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:11.523257 (+ 190us) client_negotiation.cc:769] Sending connection context
0810 14:45:11.523421 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 14:45:11.523676 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9027,"mutex_wait_us":362,"thread_start_us":8851,"threads_started":1}
I20260810 14:45:11.526283 900 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:11.527601 1712 heartbeater.cc:347] Connected to a master server at 127.24.215.190:41267
I20260810 14:45:11.527830 1711 heartbeater.cc:347] Connected to a master server at 127.24.215.188:35031
I20260810 14:45:11.528019 1712 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:11.528194 1713 heartbeater.cc:347] Connected to a master server at 127.24.215.189:36703
I20260810 14:45:11.528332 1713 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:11.528028 1711 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:11.529400 1712 heartbeater.cc:511] Master 127.24.215.190:41267 requested a full tablet report, sending...
I20260810 14:45:11.529407 1713 heartbeater.cc:511] Master 127.24.215.189:36703 requested a full tablet report, sending...
I20260810 14:45:11.530082 1711 heartbeater.cc:511] Master 127.24.215.188:35031 requested a full tablet report, sending...
I20260810 14:45:11.531220 363 ts_manager.cc:194] Re-registered known tserver with Master: 05be3da9c7e74b218e6025789e323cb9 (127.24.215.130:35709)
I20260810 14:45:11.531414 1330 ts_manager.cc:194] Registered new tserver with Master: 05be3da9c7e74b218e6025789e323cb9 (127.24.215.130:35709)
I20260810 14:45:11.531422 1091 ts_manager.cc:194] Registered new tserver with Master: 05be3da9c7e74b218e6025789e323cb9 (127.24.215.130:35709)
I20260810 14:45:11.532807 363 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:49527
I20260810 14:45:11.621806 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 750
I20260810 14:45:11.643548 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:34477
--local_ip_for_outbound_sockets=127.24.215.131
--tserver_master_addrs=127.24.215.190:41267
--webserver_port=45497
--webserver_interface=127.24.215.131
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.131.keytab
--principal=oryx/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34419
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.24.215.190:41267,127.24.215.189:36703
--tserver_master_addrs=127.24.215.190:41267,127.24.215.189:36703,127.24.215.188:35031 with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:45:11.907181 1547 heartbeater.cc:503] Master 127.24.215.190:41267 was elected leader, sending a full tablet report...
W20260810 14:45:11.985668 1726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:11.986117 1726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:11.986315 1726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:11.993835 1726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:11.993992 1726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:11.994161 1726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:45:12.004267 1726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34419
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.131.keytab
--principal=oryx/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:34477
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=45497
--tserver_master_addrs=127.24.215.190:41267,127.24.215.189:36703,127.24.215.188:35031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:12.005710 1726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:12.007218 1726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:12.021082 1734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:12.021103 1737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:12.021706 1733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:12.022195 1726 server_base.cc:1034] running on GCE node
I20260810 14:45:12.022796 1726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:12.024052 1726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:12.025290 1726 hybrid_clock.cc:648] HybridClock initialized: now 1786373112025245 us; error 45 us; skew 500 ppm
I20260810 14:45:12.025665 1726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:12 dist-test-slave-vkjp krb5kdc[313](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373112, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:12.029492 1726 init.cc:377] Logged in from keytab as oryx/127.24.215.131@KRBTEST.COM (short username oryx)
I20260810 14:45:12.031049 1726 webserver.cc:460] Webserver started at http://127.24.215.131:45497/ using document root <none> and password file <none>
I20260810 14:45:12.031821 1726 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:12.031994 1726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:12.037372 1726 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 14:45:12.040071 1746 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:12.041442 1726 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:45:12.041615 1726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "b522cc31bd6141558da804220a6775fd"
format_stamp: "Formatted at 2026-08-10 14:45:07 on dist-test-slave-vkjp"
server_key: "f02b62df140e68da799724cbaf005260"
server_key_iv: "96541e541211b71286cc8db59f7924b6"
server_key_version: "encryptionkey@0"
I20260810 14:45:12.042698 1726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:12.079315 1726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:12.084213 1726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:12.084528 1726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:12.085733 1726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:12.087226 1726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:12.087317 1726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:12.087409 1726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:12.087466 1726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:12.121591 1726 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:34477
I20260810 14:45:12.121731 1900 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:34477 every 8 connection(s)
I20260810 14:45:12.123224 1726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:45:12.126591 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 1726
Aug 10 14:45:12 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373112, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM
Aug 10 14:45:12 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373112, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.189@KRBTEST.COM
I20260810 14:45:12.175724 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.127722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:58357 (local address 127.24.215.190:41267)
0810 14:45:12.151022 (+ 23300us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:12.151028 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:12.151046 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:45:12.151135 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:12.151139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:12.151217 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:12.152659 (+ 1442us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:12.154406 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.155764 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.159014 (+ 3250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.159763 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.159878 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.165663 (+ 5785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:12.165673 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:12.165681 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:12.165734 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:12.171152 (+ 5418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.172229 (+ 1077us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.172236 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.172243 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.172457 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.173032 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.173039 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.173044 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.173446 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:12.173622 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:12.174532 (+ 910us) server_negotiation.cc:299] Negotiation successful
0810 14:45:12.174834 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23162,"thread_start_us":23029,"threads_started":1}
I20260810 14:45:12.176965 1916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.127533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36703 (local address 127.24.215.131:37667)
0810 14:45:12.150250 (+ 22717us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:12.150265 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:12.150413 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:12.151326 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:12.151341 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:12.153750 (+ 2409us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:12.154302 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:12.154323 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.155977 (+ 1654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.155984 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:12.158845 (+ 2861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:12.158860 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.160021 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.160031 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:12.160357 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.161545 (+ 1188us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:12.161575 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:12.168572 (+ 6997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:12.172840 (+ 4268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:12.172844 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:12.172848 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:12.173430 (+ 582us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:12.174217 (+ 787us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:12.174234 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:12.174237 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:12.174406 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:12.175072 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:12.175077 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:12.175269 (+ 192us) client_negotiation.cc:769] Sending connection context
0810 14:45:12.175460 (+ 191us) client_negotiation.cc:240] Negotiation successful
0810 14:45:12.175768 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22606,"mutex_wait_us":223,"spinlock_wait_cycles":19712,"thread_start_us":18818,"threads_started":1}
I20260810 14:45:12.177558 1912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.127036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.131:58357)
0810 14:45:12.146859 (+ 19823us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:12.146931 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:12.149575 (+ 2644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:12.152278 (+ 2703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:12.152282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:12.152913 (+ 631us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:12.154149 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:12.154208 (+ 59us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.156234 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.156240 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:12.158845 (+ 2605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:12.158860 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.160232 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.160239 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:12.160404 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.161399 (+ 995us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:12.161477 (+ 78us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:12.165345 (+ 3868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:12.171359 (+ 6014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:12.171370 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:12.171387 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:12.172110 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:12.172601 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:12.172609 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:12.172613 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:12.172939 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:12.173589 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:12.173597 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:12.174060 (+ 463us) client_negotiation.cc:769] Sending connection context
0810 14:45:12.174395 (+ 335us) client_negotiation.cc:240] Negotiation successful
0810 14:45:12.174996 (+ 601us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19472,"mutex_wait_us":314,"spinlock_wait_cycles":20736,"thread_start_us":24374,"threads_started":1}
I20260810 14:45:12.179234 1911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.127291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:37667 (local address 127.24.215.189:36703)
0810 14:45:12.147621 (+ 20330us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:12.147627 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:12.150359 (+ 2732us) server_negotiation.cc:407] Connection header received
0810 14:45:12.150554 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:12.150562 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:12.150685 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:12.150822 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:12.154514 (+ 3692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.155766 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.159021 (+ 3255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.159736 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.161089 (+ 1353us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.168749 (+ 7660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:12.168755 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:12.168760 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:12.168834 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:12.172693 (+ 3859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.173863 (+ 1170us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.173868 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.173872 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.174035 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.174573 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.174577 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.174579 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.174936 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:12.178304 (+ 3368us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:12.178470 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 14:45:12.178744 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20210,"thread_start_us":20099,"threads_started":1}
I20260810 14:45:12.180299 1902 heartbeater.cc:347] Connected to a master server at 127.24.215.190:41267
I20260810 14:45:12.180649 1902 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:12.181442 1902 heartbeater.cc:511] Master 127.24.215.190:41267 requested a full tablet report, sending...
I20260810 14:45:12.182406 1903 heartbeater.cc:347] Connected to a master server at 127.24.215.189:36703
I20260810 14:45:12.182533 1903 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:12.183054 1903 heartbeater.cc:511] Master 127.24.215.189:36703 requested a full tablet report, sending...
I20260810 14:45:12.183753 363 ts_manager.cc:194] Re-registered known tserver with Master: b522cc31bd6141558da804220a6775fd (127.24.215.131:34477)
I20260810 14:45:12.184751 363 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:58357
I20260810 14:45:12.186100 1091 ts_manager.cc:194] Registered new tserver with Master: b522cc31bd6141558da804220a6775fd (127.24.215.131:34477)
Aug 10 14:45:12 dist-test-slave-vkjp krb5kdc[313](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373112, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.188@KRBTEST.COM
I20260810 14:45:12.208231 1913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.157709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59150 (local address 127.24.215.190:41267)
0810 14:45:12.187466 (+ 29757us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:12.187474 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:12.187500 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:45:12.187563 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:12.187571 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:12.187665 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:12.187794 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:12.189874 (+ 2080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.190684 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.192810 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.193477 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.193601 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.198217 (+ 4616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:12.198224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:12.198228 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:12.198270 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:12.203693 (+ 5423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.204707 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.204715 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.204720 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.204866 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.205539 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.205546 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.205549 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.205937 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:12.206223 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:12.207624 (+ 1401us) server_negotiation.cc:299] Negotiation successful
0810 14:45:12.207878 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29604,"thread_start_us":44735,"threads_started":1}
I20260810 14:45:12.209676 1909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.126339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:54283 (local address 127.24.215.188:35031)
0810 14:45:12.142473 (+ 16134us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:12.142483 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:12.152049 (+ 9566us) server_negotiation.cc:407] Connection header received
0810 14:45:12.152211 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:12.152220 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:12.152337 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:12.152500 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:12.155367 (+ 2867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.156603 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.192706 (+ 36103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.193620 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.193794 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.201200 (+ 7406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:12.201209 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:12.201215 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:12.201261 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:12.206307 (+ 5046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.207179 (+ 872us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.207187 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.207193 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.207336 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.207832 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.207837 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.207841 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.208213 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:12.208355 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:12.208964 (+ 609us) server_negotiation.cc:299] Negotiation successful
0810 14:45:12.209212 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15958,"thread_start_us":15718,"threads_started":1}
I20260810 14:45:12.209762 1910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.128148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35031 (local address 127.24.215.131:54283)
0810 14:45:12.151930 (+ 23782us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:12.151948 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:12.152084 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:12.152731 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:12.152739 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:12.154712 (+ 1973us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:12.155129 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:12.155144 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.156804 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.156810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:12.192508 (+ 35698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:12.192529 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.194669 (+ 2140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.194678 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:12.194924 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.196885 (+ 1961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:12.196914 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:12.200838 (+ 3924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:12.206473 (+ 5635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:12.206479 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:12.206482 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:12.207016 (+ 534us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:12.207503 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:12.207508 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:12.207511 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:12.207683 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:12.208461 (+ 778us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:12.208465 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:12.208696 (+ 231us) client_negotiation.cc:769] Sending connection context
0810 14:45:12.208856 (+ 160us) client_negotiation.cc:240] Negotiation successful
0810 14:45:12.209244 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23660,"thread_start_us":21895,"threads_started":1}
I20260810 14:45:12.211053 1901 heartbeater.cc:347] Connected to a master server at 127.24.215.188:35031
I20260810 14:45:12.211280 1901 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:12.211824 1901 heartbeater.cc:511] Master 127.24.215.188:35031 requested a full tablet report, sending...
I20260810 14:45:12.213068 1330 ts_manager.cc:194] Registered new tserver with Master: b522cc31bd6141558da804220a6775fd (127.24.215.131:34477)
I20260810 14:45:12.213521 363 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 14:45:12.221261 1926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.164383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33032 (local address 127.24.215.189:36703)
0810 14:45:12.167851 (+ 3468us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:12.167857 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:12.199708 (+ 31851us) server_negotiation.cc:407] Connection header received
0810 14:45:12.199916 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:12.199924 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:12.200026 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:12.200139 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:12.203662 (+ 3523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.204976 (+ 1314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.207664 (+ 2688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.208595 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.208696 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.215106 (+ 6410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:12.215114 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:12.215119 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:12.215160 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:12.218082 (+ 2922us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.218889 (+ 807us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.218893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.218896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.219011 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.219470 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.219478 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.219480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.219893 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:12.220179 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:12.220519 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 14:45:12.220759 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3259,"thread_start_us":3620,"threads_started":1}
W20260810 14:45:12.223246 363 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:45:12.228361 1928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.164450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59274 (local address 127.24.215.188:35031)
0810 14:45:12.200412 (+ 35962us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:12.200421 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:12.200446 (+ 25us) server_negotiation.cc:407] Connection header received
0810 14:45:12.200503 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:12.200507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:12.200626 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:12.200746 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:12.203750 (+ 3004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.205078 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.207565 (+ 2487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.208327 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.208506 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.220950 (+ 12444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:12.220959 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:12.220964 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:12.221003 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:12.224547 (+ 3544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.225325 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.225330 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.225333 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.225474 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.225833 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.225837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.225839 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.226196 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:12.226494 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:12.227257 (+ 763us) server_negotiation.cc:299] Negotiation successful
0810 14:45:12.227487 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35705,"thread_start_us":35494,"threads_started":1}
I20260810 14:45:12.262356 1944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.245140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47080)
0810 14:45:12.248413 (+ 3273us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:12.248441 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:12.248639 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:12.251122 (+ 2483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:12.251130 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:12.251169 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:12.251585 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:12.251599 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.254158 (+ 2559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.254166 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:12.257519 (+ 3353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:12.257542 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.261311 (+ 3769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.261321 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:12.261611 (+ 290us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.261676 (+ 65us) client_negotiation.cc:769] Sending connection context
0810 14:45:12.261766 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 14:45:12.261919 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3141,"thread_start_us":2923,"threads_started":1}
I20260810 14:45:12.262411 1945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.245436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47080 (local address 127.24.215.129:46583)
0810 14:45:12.247778 (+ 2342us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:12.247793 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:12.248813 (+ 1020us) server_negotiation.cc:407] Connection header received
0810 14:45:12.248946 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:12.248955 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:12.250792 (+ 1837us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:12.251133 (+ 341us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:12.251858 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.253934 (+ 2076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.257733 (+ 3799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.261034 (+ 3301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.261314 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.261514 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:12.261935 (+ 421us) server_negotiation.cc:299] Negotiation successful
0810 14:45:12.262108 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2074,"thread_start_us":1882,"threads_started":1}
I20260810 14:45:12.267360 1946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.248535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:35709 (local address 127.0.0.1:33162)
0810 14:45:12.249207 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:12.249236 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:12.249378 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:12.260375 (+ 10997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:12.260392 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:12.260414 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:12.260846 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:12.260856 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.262641 (+ 1785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.262646 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:12.264676 (+ 2030us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:12.264689 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.266417 (+ 1728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.266423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:12.266586 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.266619 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:45:12.266726 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 14:45:12.266869 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"mutex_wait_us":1,"thread_start_us":283,"threads_started":1}
I20260810 14:45:12.267359 1947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.248734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33162 (local address 127.24.215.130:35709)
0810 14:45:12.259631 (+ 10897us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:12.259643 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:12.259680 (+ 37us) server_negotiation.cc:407] Connection header received
0810 14:45:12.259806 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:12.259817 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:12.260141 (+ 324us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:12.260383 (+ 242us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:12.261036 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.262454 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.264828 (+ 2374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.266200 (+ 1372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.266376 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.266527 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:12.266796 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 14:45:12.267044 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10669,"thread_start_us":10533,"threads_started":1}
I20260810 14:45:12.266918 1467 tablet_service.cc:1467] Processing CreateTablet for tablet c174f656a0434521aedd2a893eb77215 (DEFAULT_TABLE table=test-table [id=11a1820c508c4c8483d789f63fc16975]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:12.268502 1467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c174f656a0434521aedd2a893eb77215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:12.271607 1949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.249479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54376 (local address 127.24.215.131:34477)
0810 14:45:12.250608 (+ 1129us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:12.250616 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:12.261783 (+ 11167us) server_negotiation.cc:407] Connection header received
0810 14:45:12.262001 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:12.262009 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:12.262246 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:12.262467 (+ 221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:12.263463 (+ 996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.265132 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.269101 (+ 3969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.270417 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.270592 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.270798 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:12.271124 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 14:45:12.271291 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":872,"thread_start_us":770,"threads_started":1}
I20260810 14:45:12.271646 1948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.249167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:34477 (local address 127.0.0.1:54376)
0810 14:45:12.261582 (+ 12415us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:12.261602 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:12.261797 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:12.262785 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:12.262790 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:12.262817 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:12.263260 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:12.263271 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.266603 (+ 3332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.266607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:12.268951 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:12.268965 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.270706 (+ 1741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.270715 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:12.270894 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.270926 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 14:45:12.271024 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 14:45:12.271186 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12182,"thread_start_us":11787,"threads_started":1}
I20260810 14:45:12.272047 1635 tablet_service.cc:1467] Processing CreateTablet for tablet c174f656a0434521aedd2a893eb77215 (DEFAULT_TABLE table=test-table [id=11a1820c508c4c8483d789f63fc16975]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:12.273623 1635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c174f656a0434521aedd2a893eb77215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:12.275157 1809 tablet_service.cc:1467] Processing CreateTablet for tablet c174f656a0434521aedd2a893eb77215 (DEFAULT_TABLE table=test-table [id=11a1820c508c4c8483d789f63fc16975]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:12.276463 1809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c174f656a0434521aedd2a893eb77215. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:12.286427 1950 tablet_bootstrap.cc:492] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6: Bootstrap starting.
I20260810 14:45:12.290328 1951 tablet_bootstrap.cc:492] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9: Bootstrap starting.
I20260810 14:45:12.291422 1950 tablet_bootstrap.cc:654] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:12.291955 1952 tablet_bootstrap.cc:492] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd: Bootstrap starting.
I20260810 14:45:12.293852 1950 log.cc:824] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:12.294561 1951 tablet_bootstrap.cc:654] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:12.296286 1952 tablet_bootstrap.cc:654] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:12.296310 1951 log.cc:824] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:12.296845 1950 tablet_bootstrap.cc:492] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6: No bootstrap required, opened a new log
I20260810 14:45:12.297209 1950 ts_tablet_manager.cc:1397] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.007s
I20260810 14:45:12.297736 1952 log.cc:824] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:12.299799 1951 tablet_bootstrap.cc:492] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9: No bootstrap required, opened a new log
I20260810 14:45:12.300082 1951 ts_tablet_manager.cc:1397] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260810 14:45:12.300904 1952 tablet_bootstrap.cc:492] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd: No bootstrap required, opened a new log
I20260810 14:45:12.301259 1952 ts_tablet_manager.cc:1397] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 14:45:12.306843 1950 raft_consensus.cc:348] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } } peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } }
I20260810 14:45:12.307372 1950 raft_consensus.cc:374] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:12.307493 1950 raft_consensus.cc:729] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770eff9431a14800b13cbb540c4483a6, State: Initialized, Role: FOLLOWER
I20260810 14:45:12.308151 1950 consensus_queue.cc:260] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } } peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } }
I20260810 14:45:12.310141 1950 ts_tablet_manager.cc:1428] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6: Time spent starting tablet: real 0.013s user 0.012s sys 0.001s
I20260810 14:45:12.312016 1951 raft_consensus.cc:348] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } } peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } }
I20260810 14:45:12.312443 1951 raft_consensus.cc:374] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:12.312541 1951 raft_consensus.cc:729] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05be3da9c7e74b218e6025789e323cb9, State: Initialized, Role: FOLLOWER
I20260810 14:45:12.313004 1952 raft_consensus.cc:348] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } } peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } }
I20260810 14:45:12.313302 1951 consensus_queue.cc:260] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } } peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } }
I20260810 14:45:12.313606 1952 raft_consensus.cc:374] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:12.313719 1952 raft_consensus.cc:729] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b522cc31bd6141558da804220a6775fd, State: Initialized, Role: FOLLOWER
I20260810 14:45:12.314384 1952 consensus_queue.cc:260] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } } peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } }
I20260810 14:45:12.317309 1902 heartbeater.cc:503] Master 127.24.215.190:41267 was elected leader, sending a full tablet report...
I20260810 14:45:12.317349 1712 heartbeater.cc:503] Master 127.24.215.190:41267 was elected leader, sending a full tablet report...
I20260810 14:45:12.317584 1951 ts_tablet_manager.cc:1428] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9: Time spent starting tablet: real 0.017s user 0.017s sys 0.000s
I20260810 14:45:12.319195 1952 ts_tablet_manager.cc:1428] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd: Time spent starting tablet: real 0.018s user 0.014s sys 0.005s
W20260810 14:45:12.363837 1549 tablet.cc:2369] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:45:12.376132 1908 tablet.cc:2369] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:45:12.405861 1959 raft_consensus.cc:482] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:45:12.406267 1959 raft_consensus.cc:504] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } } peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } }
I20260810 14:45:12.407981 1959 leader_election.cc:288] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 770eff9431a14800b13cbb540c4483a6 (127.24.215.129:46583), 05be3da9c7e74b218e6025789e323cb9 (127.24.215.130:35709)
I20260810 14:45:12.414403 1910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.408225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.24.215.131:34641)
0810 14:45:12.408469 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:12.408491 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:12.408611 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:12.409124 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:12.409129 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:12.409161 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:12.409383 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:12.409407 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.410643 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.410648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:12.412463 (+ 1815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:12.412479 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.413671 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.413678 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:12.413924 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.413964 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 14:45:12.414057 (+ 93us) client_negotiation.cc:240] Negotiation successful
0810 14:45:12.414168 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 14:45:12.414529 1945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.408557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:34641 (local address 127.24.215.129:46583)
0810 14:45:12.408790 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:12.408796 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:12.408817 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:45:12.408895 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:12.408899 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:12.408995 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:12.409076 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:12.409544 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.410485 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.412631 (+ 2146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.413488 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.413607 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.413659 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:12.414113 (+ 454us) server_negotiation.cc:299] Negotiation successful
0810 14:45:12.414272 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 14:45:12.414831 1912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.408617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:35709 (local address 127.24.215.131:55781)
0810 14:45:12.408800 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:12.408817 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:12.408924 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:12.409441 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:12.409444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:12.409482 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:12.409796 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:12.409810 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.410942 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.410946 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:12.412577 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:12.412595 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.413954 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.413958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:12.414260 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.414291 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 14:45:12.414391 (+ 100us) client_negotiation.cc:240] Negotiation successful
0810 14:45:12.414525 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 14:45:12.414921 1947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.408950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:55781 (local address 127.24.215.130:35709)
0810 14:45:12.409142 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:12.409148 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:12.409164 (+ 16us) server_negotiation.cc:407] Connection header received
0810 14:45:12.409234 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:12.409239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:12.409309 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:12.409412 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:12.409949 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.410810 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.412756 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.413805 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.413968 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.414048 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:12.414437 (+ 389us) server_negotiation.cc:299] Negotiation successful
0810 14:45:12.414585 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:45:12.416244 1494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c174f656a0434521aedd2a893eb77215" candidate_uuid: "b522cc31bd6141558da804220a6775fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770eff9431a14800b13cbb540c4483a6" is_pre_election: true
I20260810 14:45:12.416285 1659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c174f656a0434521aedd2a893eb77215" candidate_uuid: "b522cc31bd6141558da804220a6775fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05be3da9c7e74b218e6025789e323cb9" is_pre_election: true
I20260810 14:45:12.416889 1494 raft_consensus.cc:2454] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b522cc31bd6141558da804220a6775fd in term 0.
I20260810 14:45:12.416899 1659 raft_consensus.cc:2454] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b522cc31bd6141558da804220a6775fd in term 0.
I20260810 14:45:12.417642 1751 leader_election.cc:302] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05be3da9c7e74b218e6025789e323cb9, b522cc31bd6141558da804220a6775fd; no voters:
I20260810 14:45:12.418028 1959 raft_consensus.cc:2781] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:45:12.418176 1959 raft_consensus.cc:482] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:45:12.418298 1959 raft_consensus.cc:3037] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:12.420953 1959 raft_consensus.cc:504] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } } peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } }
I20260810 14:45:12.421707 1959 leader_election.cc:288] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [CANDIDATE]: Term 1 election: Requested vote from peers 770eff9431a14800b13cbb540c4483a6 (127.24.215.129:46583), 05be3da9c7e74b218e6025789e323cb9 (127.24.215.130:35709)
I20260810 14:45:12.422128 1494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c174f656a0434521aedd2a893eb77215" candidate_uuid: "b522cc31bd6141558da804220a6775fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770eff9431a14800b13cbb540c4483a6"
I20260810 14:45:12.422292 1659 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c174f656a0434521aedd2a893eb77215" candidate_uuid: "b522cc31bd6141558da804220a6775fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05be3da9c7e74b218e6025789e323cb9"
I20260810 14:45:12.422317 1494 raft_consensus.cc:3037] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:12.422501 1659 raft_consensus.cc:3037] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:12.425004 1494 raft_consensus.cc:2454] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b522cc31bd6141558da804220a6775fd in term 1.
I20260810 14:45:12.425012 1659 raft_consensus.cc:2454] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b522cc31bd6141558da804220a6775fd in term 1.
I20260810 14:45:12.425487 1751 leader_election.cc:302] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05be3da9c7e74b218e6025789e323cb9, b522cc31bd6141558da804220a6775fd; no voters:
I20260810 14:45:12.425771 1959 raft_consensus.cc:2781] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:45:12.426220 1959 raft_consensus.cc:686] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [term 1 LEADER]: Becoming Leader. State: Replica: b522cc31bd6141558da804220a6775fd, State: Running, Role: LEADER
I20260810 14:45:12.426985 1959 consensus_queue.cc:237] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } } peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } }
I20260810 14:45:12.432535 361 catalog_manager.cc:5495] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd reported cstate change: term changed from 0 to 1, leader changed from <none> to b522cc31bd6141558da804220a6775fd (127.24.215.131). New cstate: current_term: 1 leader_uuid: "b522cc31bd6141558da804220a6775fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } health_report { overall_health: HEALTHY } } }
I20260810 14:45:12.493729 1949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.487850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54392 (local address 127.24.215.131:34477)
0810 14:45:12.488081 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:12.488089 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:12.488113 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:45:12.488248 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:12.488253 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:12.488370 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:12.488552 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:45:12.488924 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.490188 (+ 1264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.491779 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.492178 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.492280 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.492764 (+ 484us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:45:12.493230 (+ 466us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:45:12.493351 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:12.493459 (+ 108us) server_negotiation.cc:299] Negotiation successful
0810 14:45:12.493526 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 14:45:12.496766 1722 tablet.cc:2369] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:45:12.502025 1659 raft_consensus.cc:1261] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9 [term 1 FOLLOWER]: Refusing update from remote peer b522cc31bd6141558da804220a6775fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:45:12.502025 1494 raft_consensus.cc:1261] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6 [term 1 FOLLOWER]: Refusing update from remote peer b522cc31bd6141558da804220a6775fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:45:12.503013 1960 consensus_queue.cc:1035] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:12.503448 1959 consensus_queue.cc:1035] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:12.516770 1963 mvcc.cc:205] Tried to move back new op lower bound from 7316984268798382080 to 7316984268505722880. Current Snapshot: MvccSnapshot[applied={T|T < 7316984268798382080}]
I20260810 14:45:12.521384 1965 mvcc.cc:205] Tried to move back new op lower bound from 7316984268798382080 to 7316984268505722880. Current Snapshot: MvccSnapshot[applied={T|T < 7316984268798382080}]
I20260810 14:45:12.544251 361 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:45:12.544569 361 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:45:12.551549 361 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c5cde194e5e348ceac077b3a924912ec: Sending DeleteTablet for 3 replicas of tablet c174f656a0434521aedd2a893eb77215
I20260810 14:45:12.552975 1635 tablet_service.cc:1514] Processing DeleteTablet for tablet c174f656a0434521aedd2a893eb77215 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:12 UTC) from {username='oryx', principal='oryx/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:33162
I20260810 14:45:12.553200 1809 tablet_service.cc:1514] Processing DeleteTablet for tablet c174f656a0434521aedd2a893eb77215 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:12 UTC) from {username='oryx', principal='oryx/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:54376
I20260810 14:45:12.554315 1467 tablet_service.cc:1514] Processing DeleteTablet for tablet c174f656a0434521aedd2a893eb77215 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:12 UTC) from {username='oryx', principal='oryx/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:47080
I20260810 14:45:12.569739 1975 tablet_replica.cc:330] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd: stopping tablet replica
I20260810 14:45:12.570448 1975 raft_consensus.cc:2229] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:45:12.571434 1975 raft_consensus.cc:2258] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:12.574206 1975 ts_tablet_manager.cc:1905] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:12.583196 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 326
I20260810 14:45:12.584352 1975 ts_tablet_manager.cc:1918] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:45:12.584656 1975 log.cc:1196] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/c174f656a0434521aedd2a893eb77215
I20260810 14:45:12.585422 1975 ts_tablet_manager.cc:1939] T c174f656a0434521aedd2a893eb77215 P b522cc31bd6141558da804220a6775fd: Deleting consensus metadata
I20260810 14:45:12.587522 404 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:12.587440 347 catalog_manager.cc:4841] TS b522cc31bd6141558da804220a6775fd (127.24.215.131:34477): tablet c174f656a0434521aedd2a893eb77215 (table test-table [id=11a1820c508c4c8483d789f63fc16975]) successfully deleted
I20260810 14:45:12.588559 1974 tablet_replica.cc:330] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9: stopping tablet replica
I20260810 14:45:12.589290 1974 raft_consensus.cc:2229] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:45:12.590706 1974 raft_consensus.cc:2258] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:12.591816 1976 tablet_replica.cc:330] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6: stopping tablet replica
I20260810 14:45:12.592206 1976 raft_consensus.cc:2229] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:45:12.592792 1976 raft_consensus.cc:2258] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:12.596966 1974 ts_tablet_manager.cc:1905] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:12.600226 1976 ts_tablet_manager.cc:1905] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:12.603559 1974 ts_tablet_manager.cc:1918] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:45:12.603740 1974 log.cc:1196] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/c174f656a0434521aedd2a893eb77215
I20260810 14:45:12.604372 1974 ts_tablet_manager.cc:1939] T c174f656a0434521aedd2a893eb77215 P 05be3da9c7e74b218e6025789e323cb9: Deleting consensus metadata
I20260810 14:45:12.607551 1976 ts_tablet_manager.cc:1918] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:45:12.607739 1976 log.cc:1196] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/c174f656a0434521aedd2a893eb77215
I20260810 14:45:12.608275 1976 ts_tablet_manager.cc:1939] T c174f656a0434521aedd2a893eb77215 P 770eff9431a14800b13cbb540c4483a6: Deleting consensus metadata
I20260810 14:45:12.620194 1926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.557397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33042 (local address 127.24.215.189:36703)
0810 14:45:12.557654 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:12.557663 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:12.582754 (+ 25091us) server_negotiation.cc:407] Connection header received
0810 14:45:12.583155 (+ 401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:12.583164 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:12.583278 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:12.583369 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:12.591545 (+ 8176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.592744 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.594356 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.595004 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.595185 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.613659 (+ 18474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:12.613666 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:12.613670 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:12.613711 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:12.617013 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.617786 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.617792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.617794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.617921 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.618327 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.618342 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.618344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.618631 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:12.618862 (+ 231us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:12.619300 (+ 438us) server_negotiation.cc:299] Negotiation successful
0810 14:45:12.619547 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 14:45:12.633778 1928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.582261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59284 (local address 127.24.215.188:35031)
0810 14:45:12.582469 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:12.582476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:12.619968 (+ 37492us) server_negotiation.cc:407] Connection header received
0810 14:45:12.620155 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:12.620161 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:12.620251 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:12.620391 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:12.622448 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.623240 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.624567 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.624983 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.625136 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.628217 (+ 3081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:12.628223 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:12.628226 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:12.628259 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:12.630924 (+ 2665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.631612 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.631619 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.631623 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.631755 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.632150 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.632159 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.632161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.632503 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:12.632770 (+ 267us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:12.633113 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 14:45:12.633357 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:45:12.779771 347 catalog_manager.cc:4841] TS 770eff9431a14800b13cbb540c4483a6 (127.24.215.129:46583): tablet c174f656a0434521aedd2a893eb77215 (table test-table [id=11a1820c508c4c8483d789f63fc16975]) successfully deleted
I20260810 14:45:12.780822 347 catalog_manager.cc:4841] TS 05be3da9c7e74b218e6025789e323cb9 (127.24.215.130:35709): tablet c174f656a0434521aedd2a893eb77215 (table test-table [id=11a1820c508c4c8483d789f63fc16975]) successfully deleted
I20260810 14:45:12.781641 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 326
I20260810 14:45:12.840479 1926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.820946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33048 (local address 127.24.215.189:36703)
0810 14:45:12.821223 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:12.821231 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:12.823811 (+ 2580us) server_negotiation.cc:407] Connection header received
0810 14:45:12.824038 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:12.824048 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:12.824152 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:12.824257 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:12.826419 (+ 2162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.827381 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.829424 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.829838 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.829936 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.833538 (+ 3602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:12.833544 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:12.833548 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:12.833589 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:12.837429 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.838245 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.838249 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.838252 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.838369 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.838786 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.838793 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.838796 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.839227 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:12.839477 (+ 250us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:12.839879 (+ 402us) server_negotiation.cc:299] Negotiation successful
0810 14:45:12.840108 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260810 14:45:12.841706 1928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:12.821412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59290 (local address 127.24.215.188:35031)
0810 14:45:12.821608 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:12.821614 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:12.823553 (+ 1939us) server_negotiation.cc:407] Connection header received
0810 14:45:12.824585 (+ 1032us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:12.824591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:12.824672 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:12.824776 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:12.827047 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.828010 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.830098 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:12.830575 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:12.830705 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:12.833471 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:12.833477 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:12.833482 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:12.833524 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:12.837455 (+ 3931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.838978 (+ 1523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.838983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.838986 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.839165 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:12.839602 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:12.839608 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:12.839612 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:12.839967 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:12.840182 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:12.840983 (+ 801us) server_negotiation.cc:299] Negotiation successful
0810 14:45:12.841287 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 14:45:13.528833 1988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:13.527893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.130:41821)
0810 14:45:13.528454 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:13.528612 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":292,"threads_started":1}
W20260810 14:45:13.529404 1712 heartbeater.cc:644] Failed to heartbeat to 127.24.215.190:41267 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
I20260810 14:45:13.532904 1989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:13.532046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.131:59179)
0810 14:45:13.532550 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:13.532705 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":274,"threads_started":1}
W20260810 14:45:13.533314 1902 heartbeater.cc:644] Failed to heartbeat to 127.24.215.190:41267 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
I20260810 14:45:13.539978 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:13.539072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.129:56499)
0810 14:45:13.539602 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:13.539746 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":244,"threads_started":1}
W20260810 14:45:13.540359 1547 heartbeater.cc:644] Failed to heartbeat to 127.24.215.190:41267 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
I20260810 14:45:14.052948 1991 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c5cde194e5e348ceac077b3a924912ec)
I20260810 14:45:14.053253 1991 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } }
I20260810 14:45:14.054816 1991 leader_election.cc:288] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c5cde194e5e348ceac077b3a924912ec (127.24.215.190:41267), 0dab4669ab9e48788241ec344e283a08 (127.24.215.189:36703)
I20260810 14:45:14.056970 1992 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c5cde194e5e348ceac077b3a924912ec)
I20260810 14:45:14.057255 1992 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } }
I20260810 14:45:14.058323 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.055031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.0.0.1:59496)
0810 14:45:14.057812 (+ 2781us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.058039 (+ 227us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2688,"mutex_wait_us":30,"thread_start_us":2508,"threads_started":1}
W20260810 14:45:14.058705 1312 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
I20260810 14:45:14.059593 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.058304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.0.0.1:59498)
0810 14:45:14.058927 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.059205 (+ 278us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":301,"threads_started":1}
W20260810 14:45:14.060019 1074 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
I20260810 14:45:14.066380 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.055023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36703 (local address 127.0.0.1:33060)
0810 14:45:14.056776 (+ 1753us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.056798 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:14.056945 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:14.058135 (+ 1190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:14.058143 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:14.058181 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:14.058582 (+ 401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:14.058601 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.060442 (+ 1841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.060451 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:14.063195 (+ 2744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:14.063217 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.065323 (+ 2106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.065333 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:14.065592 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:14.065650 (+ 58us) client_negotiation.cc:769] Sending connection context
0810 14:45:14.065730 (+ 80us) client_negotiation.cc:240] Negotiation successful
0810 14:45:14.065909 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1614,"thread_start_us":1355,"threads_started":1}
I20260810 14:45:14.067569 1995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.056886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33060 (local address 127.24.215.189:36703)
0810 14:45:14.057776 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:14.057782 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:14.057803 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:45:14.057878 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:14.057883 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:14.057964 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:14.058081 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:14.058823 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.060225 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.063453 (+ 3228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.065105 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.066719 (+ 1614us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:14.066830 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:14.066917 (+ 87us) server_negotiation.cc:299] Negotiation successful
0810 14:45:14.067115 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":761,"thread_start_us":433,"threads_started":1}
I20260810 14:45:14.068228 1105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0dab4669ab9e48788241ec344e283a08" is_pre_election: true
I20260810 14:45:14.068648 1105 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9017f74124d749f0b0a6dc4e4386bf1b in term 1.
I20260810 14:45:14.069200 1992 leader_election.cc:288] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c5cde194e5e348ceac077b3a924912ec (127.24.215.190:41267), 9017f74124d749f0b0a6dc4e4386bf1b (127.24.215.188:35031)
I20260810 14:45:14.069211 1315 leader_election.cc:302] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0dab4669ab9e48788241ec344e283a08, 9017f74124d749f0b0a6dc4e4386bf1b; no voters:
I20260810 14:45:14.070111 1991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260810 14:45:14.070251 1991 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 1 FOLLOWER]: Starting leader election (detected failure of leader c5cde194e5e348ceac077b3a924912ec)
I20260810 14:45:14.070366 1991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:45:14.071920 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.071484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.0.0.1:59524)
0810 14:45:14.071639 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.071725 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 14:45:14.072119 2000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.070517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.0.0.1:59512)
0810 14:45:14.071736 (+ 1219us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.071923 (+ 187us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1113}
W20260810 14:45:14.072268 1074 leader_election.cc:334] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c5cde194e5e348ceac077b3a924912ec (127.24.215.190:41267): Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
W20260810 14:45:14.072248 1312 leader_election.cc:334] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c5cde194e5e348ceac077b3a924912ec (127.24.215.190:41267): Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
I20260810 14:45:14.074658 1991 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } }
I20260810 14:45:14.075986 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.075469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.0.0.1:59536)
0810 14:45:14.075745 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.075830 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184}
I20260810 14:45:14.075984 1105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "0dab4669ab9e48788241ec344e283a08"
I20260810 14:45:14.076205 1105 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:45:14.076370 1991 leader_election.cc:288] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [CANDIDATE]: Term 2 election: Requested vote from peers c5cde194e5e348ceac077b3a924912ec (127.24.215.190:41267), 0dab4669ab9e48788241ec344e283a08 (127.24.215.189:36703)
I20260810 14:45:14.077165 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.076741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.0.0.1:59552)
0810 14:45:14.076913 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.076998 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 14:45:14.077404 1312 leader_election.cc:334] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c5cde194e5e348ceac077b3a924912ec (127.24.215.190:41267): Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
I20260810 14:45:14.078729 1996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.059680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35031 (local address 127.0.0.1:59294)
0810 14:45:14.065517 (+ 5837us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.065549 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:14.065698 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:14.072677 (+ 6979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:14.072682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:14.072720 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:14.072916 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:14.072924 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.074521 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.074525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:14.076389 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:14.076408 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.077960 (+ 1552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.077966 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:14.078155 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:14.078181 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 14:45:14.078282 (+ 101us) client_negotiation.cc:240] Negotiation successful
0810 14:45:14.078423 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5696,"thread_start_us":11741,"threads_started":1}
I20260810 14:45:14.078768 1999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.059416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59294 (local address 127.24.215.188:35031)
0810 14:45:14.072103 (+ 12687us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:14.072121 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:14.072148 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:45:14.072292 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:14.072297 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:14.072464 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:14.072629 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:14.073100 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.074340 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.076595 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.077752 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.077936 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:14.078012 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:14.078307 (+ 295us) server_negotiation.cc:299] Negotiation successful
0810 14:45:14.078448 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11483,"thread_start_us":11343,"threads_started":1}
I20260810 14:45:14.078776 1105 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9017f74124d749f0b0a6dc4e4386bf1b in term 2.
I20260810 14:45:14.079218 1315 leader_election.cc:302] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 0dab4669ab9e48788241ec344e283a08, 9017f74124d749f0b0a6dc4e4386bf1b; no voters: c5cde194e5e348ceac077b3a924912ec
I20260810 14:45:14.079563 1991 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 2 FOLLOWER]: Leader election won for term 2
I20260810 14:45:14.079591 1342 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0dab4669ab9e48788241ec344e283a08" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" is_pre_election: true
I20260810 14:45:14.080073 1991 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [term 2 LEADER]: Becoming Leader. State: Replica: 9017f74124d749f0b0a6dc4e4386bf1b, State: Running, Role: LEADER
I20260810 14:45:14.080466 1991 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } }
I20260810 14:45:14.082348 2002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9017f74124d749f0b0a6dc4e4386bf1b. Latest consensus state: current_term: 2 leader_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } } }
I20260810 14:45:14.082837 2002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:14.082816 1076 leader_election.cc:302] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 0dab4669ab9e48788241ec344e283a08; no voters: 9017f74124d749f0b0a6dc4e4386bf1b, c5cde194e5e348ceac077b3a924912ec
I20260810 14:45:14.083140 1992 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:45:14.083762 2004 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:45:14.087929 2004 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:45:14.088766 2004 catalog_manager.cc:1259] Loaded cluster ID: 0602c91b97ca4eb48e7748d24ec2bf1b
I20260810 14:45:14.088862 2004 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:45:14.090256 2004 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:45:14.091185 2004 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: Loaded TSK: 0
I20260810 14:45:14.092082 2004 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:45:14.128554 1329 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:45:14.135592 1329 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:45:14.139592 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.139135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.0.0.1:59574)
0810 14:45:14.139294 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.139422 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 14:45:14.139712 1105 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [term 2 FOLLOWER]: Refusing update from remote peer 9017f74124d749f0b0a6dc4e4386bf1b: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260810 14:45:14.140584 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.140184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.0.0.1:59578)
0810 14:45:14.140326 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.140436 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
I20260810 14:45:14.140558 2002 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260810 14:45:14.140859 1312 consensus_peers.cc:487] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b -> Peer c5cde194e5e348ceac077b3a924912ec (127.24.215.190:41267): Couldn't send request to peer c5cde194e5e348ceac077b3a924912ec. Status: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260810 14:45:14.143199 1992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9017f74124d749f0b0a6dc4e4386bf1b. Latest consensus state: current_term: 2 leader_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } } }
I20260810 14:45:14.143473 1992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:45:14.145085 1991 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } } }
I20260810 14:45:14.145457 1991 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:14.145789 1992 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "c5cde194e5e348ceac077b3a924912ec" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41267 } } peers { permanent_uuid: "0dab4669ab9e48788241ec344e283a08" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36703 } attrs { promote: false } } peers { permanent_uuid: "9017f74124d749f0b0a6dc4e4386bf1b" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35031 } attrs { promote: false } } }
I20260810 14:45:14.146070 1992 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0dab4669ab9e48788241ec344e283a08 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:45:14.166323 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.153920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:35709 (local address 127.0.0.1:33170)
0810 14:45:14.154053 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.154073 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:14.154194 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:14.155039 (+ 845us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:14.155044 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:14.155071 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:14.155312 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:14.155325 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.157056 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.157065 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:14.163899 (+ 6834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:14.163912 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.165471 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.165477 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:14.165711 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:14.165755 (+ 44us) client_negotiation.cc:769] Sending connection context
0810 14:45:14.165831 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 14:45:14.165983 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32,"mutex_wait_us":4215}
I20260810 14:45:14.166499 2013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.154703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54406 (local address 127.24.215.131:34477)
0810 14:45:14.155220 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:14.155227 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:14.155257 (+ 30us) server_negotiation.cc:407] Connection header received
0810 14:45:14.155329 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:14.155333 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:14.155410 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:14.156242 (+ 832us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:14.157943 (+ 1701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.159422 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.164142 (+ 4720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.165362 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.165554 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:14.165638 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:14.166056 (+ 418us) server_negotiation.cc:299] Negotiation successful
0810 14:45:14.166196 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":688,"threads_started":1}
I20260810 14:45:14.166497 1994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.154314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:34477 (local address 127.0.0.1:54406)
0810 14:45:14.154643 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.154660 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:14.154778 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:14.155586 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:14.155594 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:14.155631 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:14.156057 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:14.156073 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.159629 (+ 3556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.159637 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:14.162562 (+ 2925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:14.162583 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.165637 (+ 3054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.165646 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:14.165852 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:14.165894 (+ 42us) client_negotiation.cc:769] Sending connection context
0810 14:45:14.166016 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 14:45:14.166159 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"mutex_wait_us":2}
I20260810 14:45:14.167641 2012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.154178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33170 (local address 127.24.215.130:35709)
0810 14:45:14.154628 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:14.154632 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:14.154647 (+ 15us) server_negotiation.cc:407] Connection header received
0810 14:45:14.154718 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:14.154722 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:14.154796 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:14.154983 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:14.155525 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.156853 (+ 1328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.164055 (+ 7202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.165277 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.167051 (+ 1774us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:14.167188 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:14.167240 (+ 52us) server_negotiation.cc:299] Negotiation successful
0810 14:45:14.167375 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":241,"threads_started":1}
I20260810 14:45:14.167945 1809 tablet_service.cc:1467] Processing CreateTablet for tablet be62a65342d84e6db2bc1fd635952553 (DEFAULT_TABLE table=test-table [id=7d5d48f1e3884b69999443fa54178e29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:14.168462 1809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be62a65342d84e6db2bc1fd635952553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:14.168872 1635 tablet_service.cc:1467] Processing CreateTablet for tablet be62a65342d84e6db2bc1fd635952553 (DEFAULT_TABLE table=test-table [id=7d5d48f1e3884b69999443fa54178e29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:14.169274 1635 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be62a65342d84e6db2bc1fd635952553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:14.172961 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.155084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47096)
0810 14:45:14.163593 (+ 8509us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.163608 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:14.163749 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:14.166323 (+ 2574us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:14.166328 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:14.166355 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:14.166715 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:14.166732 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.168268 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.168279 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:14.170372 (+ 2093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:14.170413 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.172139 (+ 1726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.172145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:14.172324 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:14.172360 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 14:45:14.172466 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 14:45:14.172597 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8254,"thread_start_us":8047,"threads_started":1}
I20260810 14:45:14.173655 2016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.155369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47096 (local address 127.24.215.129:46583)
0810 14:45:14.165995 (+ 10626us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:14.166000 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:14.166017 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:45:14.166088 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:14.166093 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:14.166166 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:14.166897 (+ 731us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:14.167064 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.168079 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.170583 (+ 2504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.171895 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.172081 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:14.172174 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:14.173184 (+ 1010us) server_negotiation.cc:299] Negotiation successful
0810 14:45:14.173333 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10496,"thread_start_us":10393,"threads_started":1}
I20260810 14:45:14.174500 1467 tablet_service.cc:1467] Processing CreateTablet for tablet be62a65342d84e6db2bc1fd635952553 (DEFAULT_TABLE table=test-table [id=7d5d48f1e3884b69999443fa54178e29]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:14.174911 1467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet be62a65342d84e6db2bc1fd635952553. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:14.175606 2017 tablet_bootstrap.cc:492] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd: Bootstrap starting.
I20260810 14:45:14.175758 2019 tablet_bootstrap.cc:492] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9: Bootstrap starting.
I20260810 14:45:14.177574 2019 tablet_bootstrap.cc:654] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:14.177574 2017 tablet_bootstrap.cc:654] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:14.180490 2019 tablet_bootstrap.cc:492] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9: No bootstrap required, opened a new log
I20260810 14:45:14.180531 2017 tablet_bootstrap.cc:492] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd: No bootstrap required, opened a new log
I20260810 14:45:14.180649 2019 ts_tablet_manager.cc:1397] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260810 14:45:14.180649 2017 ts_tablet_manager.cc:1397] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260810 14:45:14.180985 2022 tablet_bootstrap.cc:492] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6: Bootstrap starting.
I20260810 14:45:14.181505 2019 raft_consensus.cc:348] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } } peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } }
I20260810 14:45:14.181527 2017 raft_consensus.cc:348] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } } peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } }
I20260810 14:45:14.181753 2019 raft_consensus.cc:374] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:14.181819 2017 raft_consensus.cc:374] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:14.181860 2019 raft_consensus.cc:729] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05be3da9c7e74b218e6025789e323cb9, State: Initialized, Role: FOLLOWER
I20260810 14:45:14.181939 2017 raft_consensus.cc:729] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b522cc31bd6141558da804220a6775fd, State: Initialized, Role: FOLLOWER
I20260810 14:45:14.182196 2019 consensus_queue.cc:260] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } } peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } }
I20260810 14:45:14.182196 2017 consensus_queue.cc:260] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } } peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } }
I20260810 14:45:14.182782 2022 tablet_bootstrap.cc:654] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:14.183182 2019 ts_tablet_manager.cc:1428] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 14:45:14.187549 1711 heartbeater.cc:503] Master 127.24.215.188:35031 was elected leader, sending a full tablet report...
I20260810 14:45:14.189980 2029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.183841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.130:57013)
0810 14:45:14.188573 (+ 4732us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.188754 (+ 181us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4583,"thread_start_us":4451,"threads_started":1}
I20260810 14:45:14.190877 2017 ts_tablet_manager.cc:1428] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd: Time spent starting tablet: real 0.010s user 0.006s sys 0.002s
I20260810 14:45:14.193794 1901 heartbeater.cc:503] Master 127.24.215.188:35031 was elected leader, sending a full tablet report...
I20260810 14:45:14.194646 2029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.193945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.130:60491)
0810 14:45:14.194398 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.194503 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319}
I20260810 14:45:14.197593 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.191370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.131:39957)
0810 14:45:14.197218 (+ 5848us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.197365 (+ 147us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5752,"thread_start_us":5645,"threads_started":1}
I20260810 14:45:14.198700 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.198237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.131:47401)
0810 14:45:14.198416 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.198534 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 14:45:14.199396 2022 tablet_bootstrap.cc:492] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6: No bootstrap required, opened a new log
I20260810 14:45:14.199584 2022 ts_tablet_manager.cc:1397] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.014s
I20260810 14:45:14.200332 2022 raft_consensus.cc:348] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } } peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } }
I20260810 14:45:14.200570 2022 raft_consensus.cc:374] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:14.200714 2022 raft_consensus.cc:729] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 770eff9431a14800b13cbb540c4483a6, State: Initialized, Role: FOLLOWER
I20260810 14:45:14.200997 2022 consensus_queue.cc:260] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } } peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } }
I20260810 14:45:14.201830 2022 ts_tablet_manager.cc:1428] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260810 14:45:14.203816 1546 heartbeater.cc:503] Master 127.24.215.188:35031 was elected leader, sending a full tablet report...
I20260810 14:45:14.204253 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.202103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.129:33097)
0810 14:45:14.203754 (+ 1651us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.203952 (+ 198us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1509,"thread_start_us":1369,"threads_started":1}
I20260810 14:45:14.206122 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.205785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.129:50973)
0810 14:45:14.205891 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.205959 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":26}
I20260810 14:45:14.293342 2024 raft_consensus.cc:482] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:45:14.293610 2024 raft_consensus.cc:504] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } } peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } }
I20260810 14:45:14.294299 2024 leader_election.cc:288] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 05be3da9c7e74b218e6025789e323cb9 (127.24.215.130:35709), 770eff9431a14800b13cbb540c4483a6 (127.24.215.129:46583)
I20260810 14:45:14.294874 1658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be62a65342d84e6db2bc1fd635952553" candidate_uuid: "b522cc31bd6141558da804220a6775fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05be3da9c7e74b218e6025789e323cb9" is_pre_election: true
I20260810 14:45:14.294963 1494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be62a65342d84e6db2bc1fd635952553" candidate_uuid: "b522cc31bd6141558da804220a6775fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770eff9431a14800b13cbb540c4483a6" is_pre_election: true
I20260810 14:45:14.295224 1658 raft_consensus.cc:2454] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b522cc31bd6141558da804220a6775fd in term 0.
I20260810 14:45:14.295260 1494 raft_consensus.cc:2454] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b522cc31bd6141558da804220a6775fd in term 0.
I20260810 14:45:14.295748 1751 leader_election.cc:302] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05be3da9c7e74b218e6025789e323cb9, b522cc31bd6141558da804220a6775fd; no voters:
I20260810 14:45:14.296021 2024 raft_consensus.cc:2781] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:45:14.296154 2024 raft_consensus.cc:482] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:45:14.296252 2024 raft_consensus.cc:3037] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:14.298283 2024 raft_consensus.cc:504] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } } peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } }
I20260810 14:45:14.298784 2024 leader_election.cc:288] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [CANDIDATE]: Term 1 election: Requested vote from peers 05be3da9c7e74b218e6025789e323cb9 (127.24.215.130:35709), 770eff9431a14800b13cbb540c4483a6 (127.24.215.129:46583)
I20260810 14:45:14.299283 1658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be62a65342d84e6db2bc1fd635952553" candidate_uuid: "b522cc31bd6141558da804220a6775fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "05be3da9c7e74b218e6025789e323cb9"
I20260810 14:45:14.299362 1494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "be62a65342d84e6db2bc1fd635952553" candidate_uuid: "b522cc31bd6141558da804220a6775fd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "770eff9431a14800b13cbb540c4483a6"
I20260810 14:45:14.299469 1658 raft_consensus.cc:3037] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:14.299594 1494 raft_consensus.cc:3037] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:14.301534 1658 raft_consensus.cc:2454] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b522cc31bd6141558da804220a6775fd in term 1.
I20260810 14:45:14.301622 1494 raft_consensus.cc:2454] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b522cc31bd6141558da804220a6775fd in term 1.
I20260810 14:45:14.301937 1751 leader_election.cc:302] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 05be3da9c7e74b218e6025789e323cb9, b522cc31bd6141558da804220a6775fd; no voters:
I20260810 14:45:14.302214 2024 raft_consensus.cc:2781] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:45:14.302731 2024 raft_consensus.cc:686] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [term 1 LEADER]: Becoming Leader. State: Replica: b522cc31bd6141558da804220a6775fd, State: Running, Role: LEADER
I20260810 14:45:14.303041 2024 consensus_queue.cc:237] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } } peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } }
I20260810 14:45:14.303877 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.303388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.131:42767)
0810 14:45:14.303584 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.303708 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260810 14:45:14.305979 1329 catalog_manager.cc:5495] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd reported cstate change: term changed from 0 to 1, leader changed from <none> to b522cc31bd6141558da804220a6775fd (127.24.215.131). New cstate: current_term: 1 leader_uuid: "b522cc31bd6141558da804220a6775fd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b522cc31bd6141558da804220a6775fd" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34477 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 } health_report { overall_health: UNKNOWN } } }
I20260810 14:45:14.357492 2013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.353081 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54414 (local address 127.24.215.131:34477)
0810 14:45:14.353257 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:14.353262 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:14.353309 (+ 47us) server_negotiation.cc:407] Connection header received
0810 14:45:14.353503 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:14.353507 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:14.353574 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:14.353650 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:45:14.353982 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.354682 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.355951 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.356408 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.356510 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:14.356855 (+ 345us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:45:14.356994 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:45:14.357089 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:14.357194 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 14:45:14.357273 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 14:45:14.360523 1494 raft_consensus.cc:1261] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6 [term 1 FOLLOWER]: Refusing update from remote peer b522cc31bd6141558da804220a6775fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:45:14.360658 1658 raft_consensus.cc:1261] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9 [term 1 FOLLOWER]: Refusing update from remote peer b522cc31bd6141558da804220a6775fd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:45:14.361227 2037 consensus_queue.cc:1035] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "770eff9431a14800b13cbb540c4483a6" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:14.361617 2024 consensus_queue.cc:1035] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [LEADER]: Connected to new peer: Peer: permanent_uuid: "05be3da9c7e74b218e6025789e323cb9" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35709 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:14.364075 2029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.363512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.130:34791)
0810 14:45:14.363741 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.363863 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
I20260810 14:45:14.364230 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.363717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.129:59509)
0810 14:45:14.363927 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.364032 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116}
I20260810 14:45:14.371034 2041 mvcc.cc:205] Tried to move back new op lower bound from 7316984276415070208 to 7316984276187865088. Current Snapshot: MvccSnapshot[applied={T|T < 7316984276415070208}]
I20260810 14:45:14.371171 2034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.370586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.129:51963)
0810 14:45:14.370859 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.370959 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130}
I20260810 14:45:14.371287 2031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.370855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.131:58123)
0810 14:45:14.371012 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.371125 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260810 14:45:14.376626 2029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.376198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.24.215.130:43483)
0810 14:45:14.376369 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.376468 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 14:45:14.380267 1330 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:45:14.380614 1330 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:45:14.387972 1330 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9017f74124d749f0b0a6dc4e4386bf1b: Sending DeleteTablet for 3 replicas of tablet be62a65342d84e6db2bc1fd635952553
I20260810 14:45:14.389525 1467 tablet_service.cc:1514] Processing DeleteTablet for tablet be62a65342d84e6db2bc1fd635952553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:14 UTC) from {username='oryx', principal='oryx/127.24.215.188@KRBTEST.COM'} at 127.0.0.1:47096
I20260810 14:45:14.389744 1466 tablet_service.cc:1514] Processing DeleteTablet for tablet be62a65342d84e6db2bc1fd635952553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:14 UTC) from {username='oryx', principal='oryx/127.24.215.188@KRBTEST.COM'} at 127.0.0.1:47096
I20260810 14:45:14.390180 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 1294
I20260810 14:45:14.390532 1635 tablet_service.cc:1514] Processing DeleteTablet for tablet be62a65342d84e6db2bc1fd635952553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:14 UTC) from {username='oryx', principal='oryx/127.24.215.188@KRBTEST.COM'} at 127.0.0.1:33170
I20260810 14:45:14.391217 1634 tablet_service.cc:1514] Processing DeleteTablet for tablet be62a65342d84e6db2bc1fd635952553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:14 UTC) from {username='oryx', principal='oryx/127.24.215.188@KRBTEST.COM'} at 127.0.0.1:33170
I20260810 14:45:14.392073 1809 tablet_service.cc:1514] Processing DeleteTablet for tablet be62a65342d84e6db2bc1fd635952553 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:14 UTC) from {username='oryx', principal='oryx/127.24.215.188@KRBTEST.COM'} at 127.0.0.1:54406
I20260810 14:45:14.404102 2053 tablet_replica.cc:330] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9: stopping tablet replica
I20260810 14:45:14.404634 2055 tablet_replica.cc:330] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd: stopping tablet replica
I20260810 14:45:14.404846 2055 raft_consensus.cc:2229] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:45:14.405296 2055 raft_consensus.cc:2258] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:14.405699 2053 raft_consensus.cc:2229] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:45:14.406042 2053 raft_consensus.cc:2258] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:14.406626 2051 tablet_replica.cc:330] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6: stopping tablet replica
I20260810 14:45:14.406837 2051 raft_consensus.cc:2229] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:45:14.406915 2055 ts_tablet_manager.cc:1905] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:14.407191 2051 raft_consensus.cc:2258] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:14.407307 2053 ts_tablet_manager.cc:1905] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:14.408427 2051 ts_tablet_manager.cc:1905] T be62a65342d84e6db2bc1fd635952553 P 770eff9431a14800b13cbb540c4483a6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:14.411667 2016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.391597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47108 (local address 127.24.215.129:46583)
0810 14:45:14.391806 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:14.391812 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:14.393319 (+ 1507us) server_negotiation.cc:407] Connection header received
0810 14:45:14.393604 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:14.393609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:14.393683 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:14.393749 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:14.395619 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.396831 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.398674 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.399365 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.399467 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:14.402932 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:14.402947 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:14.402968 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:14.403010 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:14.407480 (+ 4470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:14.408426 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:14.408435 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:14.408439 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:14.408595 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:14.409083 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:14.409087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:14.409091 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:14.409547 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:14.410934 (+ 1387us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:14.411105 (+ 171us) server_negotiation.cc:299] Negotiation successful
0810 14:45:14.411320 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260810 14:45:14.412361 1541 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:14.414542 2053 ts_tablet_manager.cc:1918] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:45:14.414758 2053 log.cc:1196] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/be62a65342d84e6db2bc1fd635952553
I20260810 14:45:14.415138 2055 ts_tablet_manager.cc:1918] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:45:14.415293 2055 log.cc:1196] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/be62a65342d84e6db2bc1fd635952553
I20260810 14:45:14.415369 2053 ts_tablet_manager.cc:1939] T be62a65342d84e6db2bc1fd635952553 P 05be3da9c7e74b218e6025789e323cb9: Deleting consensus metadata
I20260810 14:45:14.415751 2055 ts_tablet_manager.cc:1939] T be62a65342d84e6db2bc1fd635952553 P b522cc31bd6141558da804220a6775fd: Deleting consensus metadata
I20260810 14:45:14.416783 1315 catalog_manager.cc:4841] TS 05be3da9c7e74b218e6025789e323cb9 (127.24.215.130:35709): tablet be62a65342d84e6db2bc1fd635952553 (table test-table [id=7d5d48f1e3884b69999443fa54178e29]) successfully deleted
I20260810 14:45:14.417093 1315 catalog_manager.cc:4841] TS b522cc31bd6141558da804220a6775fd (127.24.215.131:34477): tablet be62a65342d84e6db2bc1fd635952553 (table test-table [id=7d5d48f1e3884b69999443fa54178e29]) successfully deleted
W20260810 14:45:14.417552 1315 catalog_manager.cc:4808] TS 05be3da9c7e74b218e6025789e323cb9 (127.24.215.130:35709): delete failed for tablet be62a65342d84e6db2bc1fd635952553 because the tablet was not found. No further retry: Not found: Tablet not found: be62a65342d84e6db2bc1fd635952553
I20260810 14:45:14.513988 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 1294
I20260810 14:45:14.537501 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 1558
I20260810 14:45:14.538419 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.537888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47114)
0810 14:45:14.538100 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.538223 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 14:45:14.538718 1315 catalog_manager.cc:4584] TS 770eff9431a14800b13cbb540c4483a6 (127.24.215.129:46583): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet be62a65342d84e6db2bc1fd635952553: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
I20260810 14:45:14.553002 2012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.539249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33178 (local address 127.24.215.130:35709)
0810 14:45:14.539501 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:14.539507 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:14.539524 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:45:14.539581 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:14.539585 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:14.539657 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:14.539765 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:14.541778 (+ 2013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.542577 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.544519 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.544943 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.545043 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:14.547859 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:14.547875 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:14.547902 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:14.547936 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:14.550626 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:14.551305 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:14.551312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:14.551314 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:14.551438 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:14.551717 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:14.551720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:14.551722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:14.552073 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:14.552298 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:14.552485 (+ 187us) server_negotiation.cc:299] Negotiation successful
0810 14:45:14.552651 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:45:14.553655 1705 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:14.591488 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.590953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47124)
0810 14:45:14.591180 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.591307 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 14:45:14.592461 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.592124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47126)
0810 14:45:14.592270 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.592343 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 14:45:14.599545 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.599181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47142)
0810 14:45:14.599350 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.599419 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 14:45:14.600412 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.600084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47146)
0810 14:45:14.600254 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.600317 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260810 14:45:14.647850 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 1558
I20260810 14:45:14.661880 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.661341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47162)
0810 14:45:14.661558 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.661682 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260810 14:45:14.662483 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.662014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.0.0.1:59584)
0810 14:45:14.662222 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.662329 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260810 14:45:14.663159 1993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.662604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47172)
0810 14:45:14.662849 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.662914 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260810 14:45:14.663591 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.663159 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41267 (local address 127.0.0.1:59590)
0810 14:45:14.663325 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.663438 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:41267: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260810 14:45:14.670603 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 1726
I20260810 14:45:14.675400 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.674958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47176)
0810 14:45:14.675141 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.675227 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 14:45:14.676323 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.676007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47188)
0810 14:45:14.676146 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.676224 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 14:45:14.685195 2013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.671464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54422 (local address 127.24.215.131:34477)
0810 14:45:14.671649 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:14.671657 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:14.671677 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:45:14.671920 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:14.671924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:14.671990 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:14.672073 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:14.673418 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.674182 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.676119 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:14.676596 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:14.676735 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:14.679405 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:14.679421 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:14.679440 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:14.679476 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:14.682211 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:14.682877 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:14.682887 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:14.682891 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:14.683029 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:14.683594 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:14.683601 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:14.683605 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:14.683981 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:14.684289 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:14.684600 (+ 311us) server_negotiation.cc:299] Negotiation successful
0810 14:45:14.684824 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 14:45:14.685858 1893 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:14.763120 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.762477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47196)
0810 14:45:14.762748 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.762864 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
I20260810 14:45:14.764209 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.763840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47204)
0810 14:45:14.763992 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.764078 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 14:45:14.777750 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.777186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47214)
0810 14:45:14.777404 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.777545 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 14:45:14.778800 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.778445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47228)
0810 14:45:14.778584 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.778677 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 14:45:14.807739 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 1726
I20260810 14:45:14.834822 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 911
I20260810 14:45:14.836023 1136 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:14.905818 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.905346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47240)
0810 14:45:14.905551 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.905657 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 14:45:14.906785 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.906456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47256)
0810 14:45:14.906599 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.906674 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 14:45:14.935034 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.934539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47270)
0810 14:45:14.934772 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.934856 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260810 14:45:14.936126 2015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:14.935761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46583 (local address 127.0.0.1:47280)
0810 14:45:14.935908 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:14.935985 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260810 14:45:14.957365 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 911
I20260810 14:45:14.984539 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 1040
I20260810 14:45:14.985641 1368 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:15.109004 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 1040
2026-08-10T14:45:15Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10804 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:45:15 dist-test-slave-vkjp krb5kdc[2067](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:45:15 dist-test-slave-vkjp krb5kdc[2067](info): set up 2 sockets
Aug 10 14:45:15 dist-test-slave-vkjp krb5kdc[2067](info): commencing operation
krb5kdc: starting...
W20260810 14:45:16.315321 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:45:16 dist-test-slave-vkjp krb5kdc[2067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:45:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:45:16Z Disabled control of system clock
WARNING: no policy specified for oryx/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.190.keytab.
Entry for principal oryx/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.190.keytab.
I20260810 14:45:16.550354 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:43325
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.190.keytab
--principal=oryx/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:43325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:16.882164 2083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:16.882488 2083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:16.882614 2083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:16.890678 2083 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:45:16.890808 2083 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260810 14:45:16.890856 2083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:16.890918 2083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:16.890975 2083 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:45:16.891047 2083 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:45:16.901393 2083 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:43325
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.190.keytab
--principal=oryx/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:43325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:16.902738 2083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:16.904075 2083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:16.916216 2092 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:16.916302 2090 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:16.916252 2089 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:16.917582 2083 server_base.cc:1034] running on GCE node
I20260810 14:45:16.918416 2083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:16.919862 2083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:16.921118 2083 hybrid_clock.cc:648] HybridClock initialized: now 1786373116921055 us; error 38 us; skew 500 ppm
I20260810 14:45:16.921476 2083 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:16 dist-test-slave-vkjp krb5kdc[2067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373116, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:16.924674 2083 init.cc:377] Logged in from keytab as oryx/127.24.215.190@KRBTEST.COM (short username oryx)
I20260810 14:45:16.926173 2083 webserver.cc:460] Webserver started at http://127.24.215.190:43993/ using document root <none> and password file <none>
I20260810 14:45:16.926903 2083 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:16.927031 2083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:16.927400 2083 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:16.930361 2083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "e34e7455f58e4638ae3fad80b7500e2e"
format_stamp: "Formatted at 2026-08-10 14:45:16 on dist-test-slave-vkjp"
server_key: "cacdfa52f5d81fec34b2460b27a116ee"
server_key_iv: "0035c3e770d4ba29ccb9105f654943ba"
server_key_version: "encryptionkey@0"
I20260810 14:45:16.931236 2083 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "e34e7455f58e4638ae3fad80b7500e2e"
format_stamp: "Formatted at 2026-08-10 14:45:16 on dist-test-slave-vkjp"
server_key: "cacdfa52f5d81fec34b2460b27a116ee"
server_key_iv: "0035c3e770d4ba29ccb9105f654943ba"
server_key_version: "encryptionkey@0"
I20260810 14:45:16.936496 2083 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 14:45:16.940035 2100 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:16.941526 2083 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:45:16.941728 2083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "e34e7455f58e4638ae3fad80b7500e2e"
format_stamp: "Formatted at 2026-08-10 14:45:16 on dist-test-slave-vkjp"
server_key: "cacdfa52f5d81fec34b2460b27a116ee"
server_key_iv: "0035c3e770d4ba29ccb9105f654943ba"
server_key_version: "encryptionkey@0"
I20260810 14:45:16.941964 2083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:16.960995 2083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:16.965093 2083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:16.965372 2083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:16.987344 2083 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:43325
I20260810 14:45:16.987365 2167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:43325 every 8 connection(s)
I20260810 14:45:16.988716 2083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:45:16.992460 2168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:16.993757 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 2083
I20260810 14:45:16.994010 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:45:16.994879 25438 external_mini_cluster.cc:1428] Setting key e0e7d078dff235c61e986c210d8b3cc4
I20260810 14:45:17.007407 2168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e: Bootstrap starting.
Aug 10 14:45:17 dist-test-slave-vkjp krb5kdc[2067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373116, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM
I20260810 14:45:17.011389 2168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:17.012802 2168 log.cc:824] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:17.016274 2168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e: No bootstrap required, opened a new log
I20260810 14:45:17.019528 2173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:16.996998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34342 (local address 127.24.215.190:43325)
0810 14:45:16.997844 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:16.997857 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:16.997906 (+ 49us) server_negotiation.cc:407] Connection header received
0810 14:45:16.999577 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:16.999603 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:17.000027 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:17.000352 (+ 325us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:17.002100 (+ 1748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:17.003495 (+ 1395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:17.005309 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:17.005985 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:17.006119 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:17.010862 (+ 4743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:17.010873 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:17.010894 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:17.010945 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:17.014893 (+ 3948us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:17.015626 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:17.015635 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:17.015646 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:17.015878 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:17.016220 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:17.016227 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:17.016233 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:17.017034 (+ 801us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:17.017320 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:17.017666 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 14:45:17.018109 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":398,"threads_started":1}
I20260810 14:45:17.025404 2168 raft_consensus.cc:348] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34e7455f58e4638ae3fad80b7500e2e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 43325 } }
I20260810 14:45:17.025729 2168 raft_consensus.cc:374] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:17.025837 2168 raft_consensus.cc:729] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e34e7455f58e4638ae3fad80b7500e2e, State: Initialized, Role: FOLLOWER
I20260810 14:45:17.026454 2168 consensus_queue.cc:260] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34e7455f58e4638ae3fad80b7500e2e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 43325 } }
I20260810 14:45:17.026757 2168 raft_consensus.cc:388] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:45:17.026903 2168 raft_consensus.cc:482] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:45:17.027103 2168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:17.029245 2168 raft_consensus.cc:504] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34e7455f58e4638ae3fad80b7500e2e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 43325 } }
I20260810 14:45:17.029750 2168 leader_election.cc:302] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e34e7455f58e4638ae3fad80b7500e2e; no voters:
I20260810 14:45:17.030329 2168 leader_election.cc:288] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:45:17.030499 2175 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:45:17.031241 2175 raft_consensus.cc:686] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [term 1 LEADER]: Becoming Leader. State: Replica: e34e7455f58e4638ae3fad80b7500e2e, State: Running, Role: LEADER
I20260810 14:45:17.031867 2175 consensus_queue.cc:237] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34e7455f58e4638ae3fad80b7500e2e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 43325 } }
I20260810 14:45:17.032680 2168 sys_catalog.cc:564] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:45:17.034693 2176 sys_catalog.cc:455] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e34e7455f58e4638ae3fad80b7500e2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34e7455f58e4638ae3fad80b7500e2e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 43325 } } }
I20260810 14:45:17.035037 2176 sys_catalog.cc:458] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:17.035398 2178 sys_catalog.cc:455] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e34e7455f58e4638ae3fad80b7500e2e. Latest consensus state: current_term: 1 leader_uuid: "e34e7455f58e4638ae3fad80b7500e2e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e34e7455f58e4638ae3fad80b7500e2e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 43325 } } }
I20260810 14:45:17.035810 2178 sys_catalog.cc:458] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:17.037923 2181 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:45:17.043545 2181 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:45:17.052659 2181 catalog_manager.cc:1347] Generated new cluster ID: 618c6acaffda42d591d2db0b4685f33e
I20260810 14:45:17.052783 2181 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:45:17.068706 2181 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:45:17.069970 2181 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:45:17.079314 2181 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e: Generated new TSK 0
I20260810 14:45:17.080235 2181 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.129.keytab.
Entry for principal oryx/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.129.keytab.
I20260810 14:45:17.144819 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:43325
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.129.keytab
--principal=oryx/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:17.483769 2205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:17.484027 2205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:17.484198 2205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:17.491420 2205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:17.491541 2205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:17.491681 2205 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:45:17.491802 2205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:45:17.501394 2205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.129.keytab
--principal=oryx/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.215.190:43325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:17.502664 2205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:17.504052 2205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:17.517098 2214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:17.517151 2211 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:17.517200 2212 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:17.517840 2205 server_base.cc:1034] running on GCE node
I20260810 14:45:17.518481 2205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:17.519447 2205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:17.520681 2205 hybrid_clock.cc:648] HybridClock initialized: now 1786373117520621 us; error 38 us; skew 500 ppm
I20260810 14:45:17.520975 2205 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:17 dist-test-slave-vkjp krb5kdc[2067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373117, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:17.524056 2205 init.cc:377] Logged in from keytab as oryx/127.24.215.129@KRBTEST.COM (short username oryx)
I20260810 14:45:17.525616 2205 webserver.cc:460] Webserver started at http://127.24.215.129:33039/ using document root <none> and password file <none>
I20260810 14:45:17.526413 2205 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:17.526556 2205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:17.527165 2205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:17.530575 2205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "7e892cc64bb7430aa632692d7ad94d5e"
format_stamp: "Formatted at 2026-08-10 14:45:17 on dist-test-slave-vkjp"
server_key: "e0c8078d0f4c846753863b9abfb9843f"
server_key_iv: "781bc2e9e1f546d8b9d4171b6f242d0e"
server_key_version: "encryptionkey@0"
I20260810 14:45:17.531592 2205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "7e892cc64bb7430aa632692d7ad94d5e"
format_stamp: "Formatted at 2026-08-10 14:45:17 on dist-test-slave-vkjp"
server_key: "e0c8078d0f4c846753863b9abfb9843f"
server_key_iv: "781bc2e9e1f546d8b9d4171b6f242d0e"
server_key_version: "encryptionkey@0"
I20260810 14:45:17.537588 2205 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:45:17.541322 2222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:17.542657 2205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 14:45:17.542869 2205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "7e892cc64bb7430aa632692d7ad94d5e"
format_stamp: "Formatted at 2026-08-10 14:45:17 on dist-test-slave-vkjp"
server_key: "e0c8078d0f4c846753863b9abfb9843f"
server_key_iv: "781bc2e9e1f546d8b9d4171b6f242d0e"
server_key_version: "encryptionkey@0"
I20260810 14:45:17.543147 2205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:17.565809 2205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:17.569707 2205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:17.569996 2205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:17.572762 2205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:17.572904 2205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:17.573067 2205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:17.573165 2205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:45:17 dist-test-slave-vkjp krb5kdc[2067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373117, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM
I20260810 14:45:17.595329 2233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:17.573196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:56489 (local address 127.24.215.190:43325)
0810 14:45:17.573706 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:17.573714 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:17.574913 (+ 1199us) server_negotiation.cc:407] Connection header received
0810 14:45:17.577034 (+ 2121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:17.577040 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:17.577135 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:17.577255 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:17.579297 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:17.580331 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:17.583028 (+ 2697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:17.583915 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:17.584053 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:17.588274 (+ 4221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:17.588279 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:17.588283 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:17.588317 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:17.590808 (+ 2491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:17.591836 (+ 1028us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:17.591843 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:17.591847 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:17.592041 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:17.592706 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:17.592712 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:17.592717 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:17.593253 (+ 536us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:17.593430 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:17.594652 (+ 1222us) server_negotiation.cc:299] Negotiation successful
0810 14:45:17.594931 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":270,"threads_started":1}
I20260810 14:45:17.597124 2236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:17.573835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:43325 (local address 127.24.215.129:56489)
0810 14:45:17.574729 (+ 894us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:17.574774 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:17.576735 (+ 1961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:17.577518 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:17.577526 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:17.578317 (+ 791us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:17.579060 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:17.579121 (+ 61us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:17.580498 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:17.580509 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:17.582853 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:17.582873 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:17.584154 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:17.584159 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:17.584350 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:17.585121 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:17.585169 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:17.588053 (+ 2884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:17.590954 (+ 2901us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:17.590964 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:17.590979 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:17.591710 (+ 731us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:17.592201 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:17.592208 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:17.592215 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:17.592530 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:17.593418 (+ 888us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:17.593432 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:17.594023 (+ 591us) client_negotiation.cc:769] Sending connection context
0810 14:45:17.594426 (+ 403us) client_negotiation.cc:240] Negotiation successful
0810 14:45:17.595175 (+ 749us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"thread_start_us":367,"threads_started":1}
I20260810 14:45:17.623872 2233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:17.603895 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:35059 (local address 127.24.215.190:43325)
0810 14:45:17.604105 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:17.604113 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:17.605099 (+ 986us) server_negotiation.cc:407] Connection header received
0810 14:45:17.605241 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:17.605249 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:17.605357 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:17.605502 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:17.606349 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:17.607711 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:17.609710 (+ 1999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:17.610128 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:17.610274 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:17.612783 (+ 2509us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:17.612792 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:17.612799 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:17.612845 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:17.620490 (+ 7645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:17.621488 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:17.621496 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:17.621499 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:17.621712 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:17.622278 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:17.622286 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:17.622291 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:17.622649 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:17.622813 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:17.623247 (+ 434us) server_negotiation.cc:299] Negotiation successful
0810 14:45:17.623504 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 14:45:17.623988 2294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:17.604074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:43325 (local address 127.24.215.129:35059)
0810 14:45:17.604986 (+ 912us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:17.605004 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:17.605121 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:17.605501 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:17.605504 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:17.605931 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:17.606213 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:17.606221 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:17.607899 (+ 1678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:17.607908 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:17.609534 (+ 1626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:17.609547 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:17.610322 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:17.610328 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:17.610518 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:17.611286 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:17.611316 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:17.612513 (+ 1197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:17.620644 (+ 8131us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:17.620650 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:17.620658 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:17.621334 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:17.621891 (+ 557us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:17.621899 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:17.621906 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:17.622080 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:17.622779 (+ 699us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:17.622783 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:17.622976 (+ 193us) client_negotiation.cc:769] Sending connection context
0810 14:45:17.623150 (+ 174us) client_negotiation.cc:240] Negotiation successful
0810 14:45:17.623523 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":747,"thread_start_us":600,"threads_started":1}
I20260810 14:45:17.634613 2205 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:41597
I20260810 14:45:17.634655 2364 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:41597 every 8 connection(s)
I20260810 14:45:17.635921 2205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:45:17.638041 2366 heartbeater.cc:347] Connected to a master server at 127.24.215.190:43325
I20260810 14:45:17.638283 2366 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:17.638922 2366 heartbeater.cc:511] Master 127.24.215.190:43325 requested a full tablet report, sending...
I20260810 14:45:17.640872 2119 ts_manager.cc:194] Registered new tserver with Master: 7e892cc64bb7430aa632692d7ad94d5e (127.24.215.129:41597)
I20260810 14:45:17.642395 2119 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:56489
I20260810 14:45:17.642684 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 2205
I20260810 14:45:17.642907 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:45:17.643671 25438 external_mini_cluster.cc:1428] Setting key cae22da72566ae4d79ac11b09593ae15
WARNING: no policy specified for oryx/127.24.215.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.215.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.130.keytab.
Entry for principal oryx/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.130.keytab.
I20260810 14:45:17.694257 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:43325
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.130.keytab
--principal=oryx/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:18.030846 2370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:18.031152 2370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:18.031327 2370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:18.039026 2370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:18.039165 2370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:18.039314 2370 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:45:18.039436 2370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:45:18.049846 2370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.130.keytab
--principal=oryx/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.215.190:43325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:18.051216 2370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:18.052608 2370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:18.065533 2375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:18.065598 2378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:18.065564 2381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:18.066771 2370 server_base.cc:1034] running on GCE node
I20260810 14:45:18.067469 2370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:18.068429 2370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:18.069667 2370 hybrid_clock.cc:648] HybridClock initialized: now 1786373118069608 us; error 40 us; skew 500 ppm
I20260810 14:45:18.069984 2370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:18 dist-test-slave-vkjp krb5kdc[2067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373118, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:18.073164 2370 init.cc:377] Logged in from keytab as oryx/127.24.215.130@KRBTEST.COM (short username oryx)
I20260810 14:45:18.074604 2370 webserver.cc:460] Webserver started at http://127.24.215.130:45901/ using document root <none> and password file <none>
I20260810 14:45:18.075384 2370 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:18.075531 2370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:18.075861 2370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:18.078792 2370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "76c1ea19690a4018870aea9314f752d8"
format_stamp: "Formatted at 2026-08-10 14:45:18 on dist-test-slave-vkjp"
server_key: "3a8a46944cb7d17a38c6e7fc7ce20d51"
server_key_iv: "37322a62be1d776e0151679f3fb882bf"
server_key_version: "encryptionkey@0"
I20260810 14:45:18.079638 2370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "76c1ea19690a4018870aea9314f752d8"
format_stamp: "Formatted at 2026-08-10 14:45:18 on dist-test-slave-vkjp"
server_key: "3a8a46944cb7d17a38c6e7fc7ce20d51"
server_key_iv: "37322a62be1d776e0151679f3fb882bf"
server_key_version: "encryptionkey@0"
I20260810 14:45:18.084751 2370 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:45:18.088131 2390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:18.089354 2370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:45:18.089520 2370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "76c1ea19690a4018870aea9314f752d8"
format_stamp: "Formatted at 2026-08-10 14:45:18 on dist-test-slave-vkjp"
server_key: "3a8a46944cb7d17a38c6e7fc7ce20d51"
server_key_iv: "37322a62be1d776e0151679f3fb882bf"
server_key_version: "encryptionkey@0"
I20260810 14:45:18.089740 2370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:18.120184 2370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:18.124178 2370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:18.124472 2370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:18.127267 2370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:18.127372 2370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:18.127523 2370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:18.127591 2370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:45:18 dist-test-slave-vkjp krb5kdc[2067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373118, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM
I20260810 14:45:18.151633 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.128114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:47241 (local address 127.24.215.190:43325)
0810 14:45:18.128615 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:18.128623 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:18.130036 (+ 1413us) server_negotiation.cc:407] Connection header received
0810 14:45:18.133255 (+ 3219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:18.133260 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:18.133332 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:18.133485 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:18.135131 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.136205 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.139741 (+ 3536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.140274 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.140420 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.144785 (+ 4365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:18.144790 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:18.144794 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:18.144827 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:18.148287 (+ 3460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:18.149226 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:18.149232 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:18.149236 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:18.149368 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:18.149816 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:18.149821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:18.149824 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:18.150096 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:18.150266 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:18.150982 (+ 716us) server_negotiation.cc:299] Negotiation successful
0810 14:45:18.151239 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":240,"threads_started":1}
I20260810 14:45:18.153231 2402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.128800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:43325 (local address 127.24.215.130:47241)
0810 14:45:18.129844 (+ 1044us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:18.129906 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:18.132883 (+ 2977us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:18.133742 (+ 859us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:18.133750 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:18.134353 (+ 603us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:18.134922 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.134941 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.136408 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.136416 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.139557 (+ 3141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.139581 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.140545 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.140552 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.140756 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.141438 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:18.141461 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:18.144551 (+ 3090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:18.148415 (+ 3864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:18.148428 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:18.148448 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:18.149088 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:18.149488 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:18.149493 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:18.149496 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:18.149688 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:18.150228 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:18.150236 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:18.150610 (+ 374us) client_negotiation.cc:769] Sending connection context
0810 14:45:18.150869 (+ 259us) client_negotiation.cc:240] Negotiation successful
0810 14:45:18.151469 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":695,"thread_start_us":425,"threads_started":1}
I20260810 14:45:18.176349 2462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.160093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:43325 (local address 127.24.215.130:47983)
0810 14:45:18.161039 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:18.161055 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:18.161228 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:18.161630 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:18.161636 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:18.162499 (+ 863us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:18.163137 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.163161 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.164258 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.164264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.166527 (+ 2263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.166549 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.167418 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.167426 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.167670 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.168489 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:18.168516 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:18.170006 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:18.173605 (+ 3599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:18.173610 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:18.173615 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:18.174024 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:18.174405 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:18.174410 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:18.174414 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:18.174546 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:18.175221 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:18.175226 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:18.175464 (+ 238us) client_negotiation.cc:769] Sending connection context
0810 14:45:18.175671 (+ 207us) client_negotiation.cc:240] Negotiation successful
0810 14:45:18.176010 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":763,"thread_start_us":453,"threads_started":1}
I20260810 14:45:18.176604 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.159576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:47983 (local address 127.24.215.190:43325)
0810 14:45:18.159809 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:18.159818 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:18.161144 (+ 1326us) server_negotiation.cc:407] Connection header received
0810 14:45:18.161364 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:18.161370 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:18.161451 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:18.161578 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:18.163319 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.164077 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.166696 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.167173 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.167307 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.170198 (+ 2891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:18.170203 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:18.170207 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:18.170240 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:18.173467 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:18.174151 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:18.174157 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:18.174160 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:18.174288 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:18.174654 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:18.174672 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:18.174675 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:18.175029 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:18.175249 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:18.175806 (+ 557us) server_negotiation.cc:299] Negotiation successful
0810 14:45:18.176126 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 14:45:18.184016 2370 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:38085
I20260810 14:45:18.184038 2533 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:38085 every 8 connection(s)
I20260810 14:45:18.185165 2370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:45:18.187271 2534 heartbeater.cc:347] Connected to a master server at 127.24.215.190:43325
I20260810 14:45:18.187507 2534 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:18.187676 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 2370
I20260810 14:45:18.187870 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:45:18.188215 2534 heartbeater.cc:511] Master 127.24.215.190:43325 requested a full tablet report, sending...
I20260810 14:45:18.188333 25438 external_mini_cluster.cc:1428] Setting key 10a06cbe669dfb5012eccdd656c8277b
I20260810 14:45:18.189714 2118 ts_manager.cc:194] Registered new tserver with Master: 76c1ea19690a4018870aea9314f752d8 (127.24.215.130:38085)
I20260810 14:45:18.190675 2118 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:47241
WARNING: no policy specified for oryx/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.131.keytab.
Entry for principal oryx/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.131.keytab.
I20260810 14:45:18.242640 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:43325
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.131.keytab
--principal=oryx/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:42387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:18.582159 2538 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:18.582517 2538 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:18.582744 2538 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:18.590232 2538 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:18.590356 2538 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:18.590478 2538 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260810 14:45:18.590622 2538 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:45:18.601212 2538 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:42387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.131.keytab
--principal=oryx/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.24.215.190:43325
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:18.602492 2538 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:18.603871 2538 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:18.617900 2545 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:18.617973 2546 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:18.618352 2538 server_base.cc:1034] running on GCE node
W20260810 14:45:18.617945 2548 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:18.619385 2538 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:18.620333 2538 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:18.621564 2538 hybrid_clock.cc:648] HybridClock initialized: now 1786373118621482 us; error 57 us; skew 500 ppm
I20260810 14:45:18.621845 2538 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:18 dist-test-slave-vkjp krb5kdc[2067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373118, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:18.625118 2538 init.cc:377] Logged in from keytab as oryx/127.24.215.131@KRBTEST.COM (short username oryx)
I20260810 14:45:18.626668 2538 webserver.cc:460] Webserver started at http://127.24.215.131:44653/ using document root <none> and password file <none>
I20260810 14:45:18.627434 2538 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:18.627537 2538 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:18.627815 2538 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:18.630775 2538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "2f913dae965f4b248b25cf6105cc0f1e"
format_stamp: "Formatted at 2026-08-10 14:45:18 on dist-test-slave-vkjp"
server_key: "7a61c247269b3790d2c5b998b5564886"
server_key_iv: "50c315c437d2b94c76b39b5c22784db2"
server_key_version: "encryptionkey@0"
I20260810 14:45:18.631698 2538 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "2f913dae965f4b248b25cf6105cc0f1e"
format_stamp: "Formatted at 2026-08-10 14:45:18 on dist-test-slave-vkjp"
server_key: "7a61c247269b3790d2c5b998b5564886"
server_key_iv: "50c315c437d2b94c76b39b5c22784db2"
server_key_version: "encryptionkey@0"
I20260810 14:45:18.637393 2538 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 14:45:18.641184 2556 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:18.642585 2538 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:45:18.642750 2538 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "2f913dae965f4b248b25cf6105cc0f1e"
format_stamp: "Formatted at 2026-08-10 14:45:18 on dist-test-slave-vkjp"
server_key: "7a61c247269b3790d2c5b998b5564886"
server_key_iv: "50c315c437d2b94c76b39b5c22784db2"
server_key_version: "encryptionkey@0"
I20260810 14:45:18.642977 2538 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:18.645539 2366 heartbeater.cc:503] Master 127.24.215.190:43325 was elected leader, sending a full tablet report...
I20260810 14:45:18.663964 2538 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:18.668372 2538 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:18.668629 2538 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:18.671342 2538 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:18.671461 2538 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:18.671629 2538 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:18.671705 2538 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:45:18 dist-test-slave-vkjp krb5kdc[2067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373118, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM
I20260810 14:45:18.698552 2567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.672709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:43325 (local address 127.24.215.131:35089)
0810 14:45:18.673607 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:18.673658 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:18.675816 (+ 2158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:18.676701 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:18.676713 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:18.677522 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:18.678401 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.678438 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.680276 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.680286 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.682498 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.682514 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.683830 (+ 1316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.683844 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.684037 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.684874 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:18.684916 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:18.688484 (+ 3568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:18.692869 (+ 4385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:18.692882 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:18.692903 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:18.693710 (+ 807us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:18.694268 (+ 558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:18.694273 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:18.694277 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:18.694519 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:18.695438 (+ 919us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:18.695447 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:18.695875 (+ 428us) client_negotiation.cc:769] Sending connection context
0810 14:45:18.696161 (+ 286us) client_negotiation.cc:240] Negotiation successful
0810 14:45:18.696818 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"thread_start_us":398,"threads_started":1}
I20260810 14:45:18.700495 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.672125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:35089 (local address 127.24.215.190:43325)
0810 14:45:18.672321 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:18.672327 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:18.673811 (+ 1484us) server_negotiation.cc:407] Connection header received
0810 14:45:18.676145 (+ 2334us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:18.676151 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:18.676265 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:18.676379 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:18.678651 (+ 2272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.680065 (+ 1414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.682661 (+ 2596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.683512 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.683672 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.688817 (+ 5145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:18.688823 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:18.688826 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:18.688884 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:18.692657 (+ 3773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:18.693867 (+ 1210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:18.693873 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:18.693878 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:18.694078 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:18.694777 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:18.694782 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:18.694788 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:18.695254 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:18.699605 (+ 4351us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:18.699796 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 14:45:18.700099 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 14:45:18.715394 2538 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:41627
I20260810 14:45:18.715448 2704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:41627 every 8 connection(s)
I20260810 14:45:18.716722 2538 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:45:18.719312 2705 heartbeater.cc:347] Connected to a master server at 127.24.215.190:43325
I20260810 14:45:18.719545 2705 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:18.720227 2705 heartbeater.cc:511] Master 127.24.215.190:43325 requested a full tablet report, sending...
I20260810 14:45:18.721853 2118 ts_manager.cc:194] Registered new tserver with Master: 2f913dae965f4b248b25cf6105cc0f1e (127.24.215.131:41627)
I20260810 14:45:18.722860 2690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.708092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:43325 (local address 127.24.215.131:60913)
0810 14:45:18.708890 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:18.708916 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:18.709121 (+ 205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:18.709478 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:18.709485 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:18.710219 (+ 734us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:18.710715 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.710733 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.712067 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.712076 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.713767 (+ 1691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.713781 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.714789 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.714798 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.715073 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.715816 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:18.715833 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:18.716596 (+ 763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:18.720292 (+ 3696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:18.720298 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:18.720302 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:18.720726 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:18.721161 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:18.721165 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:18.721168 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:18.721344 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:18.721994 (+ 650us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:18.722001 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:18.722181 (+ 180us) client_negotiation.cc:769] Sending connection context
0810 14:45:18.722305 (+ 124us) client_negotiation.cc:240] Negotiation successful
0810 14:45:18.722549 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":389,"threads_started":1}
I20260810 14:45:18.722922 2118 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:35089
I20260810 14:45:18.723021 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.708079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:60913 (local address 127.24.215.190:43325)
0810 14:45:18.708298 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:18.708307 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:18.709047 (+ 740us) server_negotiation.cc:407] Connection header received
0810 14:45:18.709255 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:18.709259 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:18.709325 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:18.709417 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:18.710872 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.711896 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.713920 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.714354 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.714513 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.716801 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:18.716807 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:18.716810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:18.716868 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:18.720162 (+ 3294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:18.720846 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:18.720853 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:18.720857 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:18.721009 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:18.721474 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:18.721480 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:18.721482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:18.721824 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:18.721990 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:18.722441 (+ 451us) server_negotiation.cc:299] Negotiation successful
0810 14:45:18.722662 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 14:45:18.723695 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 2538
I20260810 14:45:18.723914 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:45:18.724463 25438 external_mini_cluster.cc:1428] Setting key 504be86d0cb11dbaf8ef93b29f7c62ac
I20260810 14:45:18.727695 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 10 14:45:18 dist-test-slave-vkjp krb5kdc[2067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373118, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 10 14:45:18 dist-test-slave-vkjp krb5kdc[2067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:18 dist-test-slave-vkjp krb5kdc[2067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 14:45:18.783798 2401 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:18.773280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34356 (local address 127.24.215.190:43325)
0810 14:45:18.773624 (+ 344us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:18.773633 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:18.773748 (+ 115us) server_negotiation.cc:407] Connection header received
0810 14:45:18.773908 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:18.773912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:18.773979 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:18.774069 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:18.775405 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.776166 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.777420 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.777828 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.777932 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.783454 (+ 5522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34356: BlockingRecv error: recv got EOF from 127.0.0.1:34356 (error 108)
Metrics: {"server-negotiator.queue_time_us":188}
Aug 10 14:45:18 dist-test-slave-vkjp krb5kdc[2067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373118, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM
I20260810 14:45:18.800001 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.786810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34358 (local address 127.24.215.190:43325)
0810 14:45:18.787045 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:18.787051 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:18.787512 (+ 461us) server_negotiation.cc:407] Connection header received
0810 14:45:18.787642 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:18.787646 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:18.787712 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:18.787801 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:18.788873 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.789645 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.790846 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.791396 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.791502 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.795244 (+ 3742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:18.795249 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:18.795253 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:18.795287 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:18.797687 (+ 2400us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:18.798316 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:18.798319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:18.798322 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:18.798448 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:18.798764 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:18.798767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:18.798769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:18.799022 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:18.799307 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:18.799540 (+ 233us) server_negotiation.cc:299] Negotiation successful
0810 14:45:18.799727 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 14:45:18.804253 2118 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34358:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:45:18.811295 2118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:45:18.836175 2729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.823829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:38085 (local address 127.0.0.1:44744)
0810 14:45:18.827029 (+ 3200us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:18.827053 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:18.827238 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:18.828843 (+ 1605us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:18.828848 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:18.828905 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:18.829352 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.829366 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.830715 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.830721 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.833202 (+ 2481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.833215 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.835012 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.835018 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.835246 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.835302 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 14:45:18.835454 (+ 152us) client_negotiation.cc:240] Negotiation successful
0810 14:45:18.835666 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3035,"mutex_wait_us":28,"spinlock_wait_cycles":11520,"thread_start_us":3023,"threads_started":1}
I20260810 14:45:18.836470 2726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.823617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:41627 (local address 127.0.0.1:35790)
0810 14:45:18.824469 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:18.824536 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:18.824764 (+ 228us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:18.828675 (+ 3911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:18.828685 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:18.828716 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:18.829257 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.829272 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.831303 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.831309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.833744 (+ 2435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.833762 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.835560 (+ 1798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.835567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.835728 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.835766 (+ 38us) client_negotiation.cc:769] Sending connection context
0810 14:45:18.835893 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 14:45:18.836060 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":531,"threads_started":1}
I20260810 14:45:18.837150 2732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.826618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40950 (local address 127.24.215.129:41597)
0810 14:45:18.828206 (+ 1588us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:18.828217 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:18.828772 (+ 555us) server_negotiation.cc:407] Connection header received
0810 14:45:18.829155 (+ 383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:18.829165 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:18.829400 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:18.829572 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:18.830444 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.831780 (+ 1336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.834268 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.835471 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.835632 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.835795 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:18.836670 (+ 875us) server_negotiation.cc:299] Negotiation successful
0810 14:45:18.836845 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1397,"thread_start_us":1255,"threads_started":1}
I20260810 14:45:18.837316 2727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.823832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35790 (local address 127.24.215.131:41627)
0810 14:45:18.827976 (+ 4144us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:18.827985 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:18.828019 (+ 34us) server_negotiation.cc:407] Connection header received
0810 14:45:18.828121 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:18.828128 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:18.828406 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:18.829234 (+ 828us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:18.829598 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.831167 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.833903 (+ 2736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.835345 (+ 1442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.835565 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.835724 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:18.836885 (+ 1161us) server_negotiation.cc:299] Negotiation successful
0810 14:45:18.837022 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3941,"thread_start_us":3718,"threads_started":1}
I20260810 14:45:18.837356 2730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.824017 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44744 (local address 127.24.215.130:38085)
0810 14:45:18.826571 (+ 2554us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:18.826580 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:18.828129 (+ 1549us) server_negotiation.cc:407] Connection header received
0810 14:45:18.828220 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:18.828226 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:18.828429 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:18.828622 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:18.829552 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.830572 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.833405 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.834797 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.834987 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.835190 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:18.836866 (+ 1676us) server_negotiation.cc:299] Negotiation successful
0810 14:45:18.837043 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2376,"thread_start_us":2564,"threads_started":1}
I20260810 14:45:18.837906 2731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.826357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:41597 (local address 127.0.0.1:40950)
0810 14:45:18.828658 (+ 2301us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:18.828677 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:18.828914 (+ 237us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:18.829831 (+ 917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:18.829837 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:18.829866 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:18.830262 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.830275 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.831957 (+ 1682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.831962 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.834128 (+ 2166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.834139 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.835953 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.835960 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.836159 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.836204 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 14:45:18.836446 (+ 242us) client_negotiation.cc:240] Negotiation successful
0810 14:45:18.837658 (+ 1212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2181,"mutex_wait_us":3,"spinlock_wait_cycles":12416,"thread_start_us":2616,"threads_started":1}
I20260810 14:45:18.841125 2446 tablet_service.cc:1467] Processing CreateTablet for tablet 8de11e5b0e614b3ba53511af7d36168d (DEFAULT_TABLE table=test-table [id=8a0a4a2d23444f13853a1933bcc82702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:18.840981 2617 tablet_service.cc:1467] Processing CreateTablet for tablet 8de11e5b0e614b3ba53511af7d36168d (DEFAULT_TABLE table=test-table [id=8a0a4a2d23444f13853a1933bcc82702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:18.841447 2283 tablet_service.cc:1467] Processing CreateTablet for tablet 8de11e5b0e614b3ba53511af7d36168d (DEFAULT_TABLE table=test-table [id=8a0a4a2d23444f13853a1933bcc82702]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:18.842505 2617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8de11e5b0e614b3ba53511af7d36168d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:18.842562 2446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8de11e5b0e614b3ba53511af7d36168d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:18.842655 2283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8de11e5b0e614b3ba53511af7d36168d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:18.855647 2733 tablet_bootstrap.cc:492] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e: Bootstrap starting.
I20260810 14:45:18.857147 2734 tablet_bootstrap.cc:492] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e: Bootstrap starting.
I20260810 14:45:18.859683 2733 tablet_bootstrap.cc:654] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:18.859776 2737 tablet_bootstrap.cc:492] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8: Bootstrap starting.
I20260810 14:45:18.860695 2734 tablet_bootstrap.cc:654] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:18.861284 2733 log.cc:824] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:18.861997 2734 log.cc:824] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:18.863034 2737 tablet_bootstrap.cc:654] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:18.864153 2737 log.cc:824] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:18.864500 2733 tablet_bootstrap.cc:492] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e: No bootstrap required, opened a new log
I20260810 14:45:18.864797 2733 ts_tablet_manager.cc:1397] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 14:45:18.865535 2734 tablet_bootstrap.cc:492] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e: No bootstrap required, opened a new log
I20260810 14:45:18.865794 2734 ts_tablet_manager.cc:1397] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 14:45:18.866634 2737 tablet_bootstrap.cc:492] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8: No bootstrap required, opened a new log
I20260810 14:45:18.866873 2737 ts_tablet_manager.cc:1397] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260810 14:45:18.873867 2737 raft_consensus.cc:348] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } }
I20260810 14:45:18.874274 2737 raft_consensus.cc:374] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:18.874372 2737 raft_consensus.cc:729] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c1ea19690a4018870aea9314f752d8, State: Initialized, Role: FOLLOWER
I20260810 14:45:18.875005 2737 consensus_queue.cc:260] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } }
I20260810 14:45:18.875619 2733 raft_consensus.cc:348] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } }
I20260810 14:45:18.876300 2534 heartbeater.cc:503] Master 127.24.215.190:43325 was elected leader, sending a full tablet report...
I20260810 14:45:18.876317 2734 raft_consensus.cc:348] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } }
I20260810 14:45:18.876732 2734 raft_consensus.cc:374] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:18.876843 2734 raft_consensus.cc:729] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e892cc64bb7430aa632692d7ad94d5e, State: Initialized, Role: FOLLOWER
I20260810 14:45:18.876863 2737 ts_tablet_manager.cc:1428] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8: Time spent starting tablet: real 0.010s user 0.008s sys 0.003s
I20260810 14:45:18.877424 2734 consensus_queue.cc:260] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } }
I20260810 14:45:18.877646 2733 raft_consensus.cc:374] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:18.877764 2733 raft_consensus.cc:729] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f913dae965f4b248b25cf6105cc0f1e, State: Initialized, Role: FOLLOWER
I20260810 14:45:18.878479 2733 consensus_queue.cc:260] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } }
I20260810 14:45:18.880594 2705 heartbeater.cc:503] Master 127.24.215.190:43325 was elected leader, sending a full tablet report...
I20260810 14:45:18.880640 2733 ts_tablet_manager.cc:1428] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e: Time spent starting tablet: real 0.016s user 0.013s sys 0.000s
I20260810 14:45:18.882743 2734 ts_tablet_manager.cc:1428] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e: Time spent starting tablet: real 0.017s user 0.013s sys 0.005s
W20260810 14:45:18.888303 2367 tablet.cc:2369] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:45:18.937188 2535 tablet.cc:2369] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:45:18.968611 2706 tablet.cc:2369] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:45:18.986099 2742 raft_consensus.cc:482] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:45:18.986414 2742 raft_consensus.cc:504] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } }
I20260810 14:45:18.988451 2742 leader_election.cc:288] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f913dae965f4b248b25cf6105cc0f1e (127.24.215.131:41627), 7e892cc64bb7430aa632692d7ad94d5e (127.24.215.129:41597)
I20260810 14:45:18.997185 2727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.989432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:45191 (local address 127.24.215.131:41627)
0810 14:45:18.989659 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:18.989668 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:18.989695 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:45:18.989888 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:18.989894 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:18.989968 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:18.990027 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:18.990722 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.992049 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.994750 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.996021 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.996152 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.996225 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:18.996780 (+ 555us) server_negotiation.cc:299] Negotiation successful
0810 14:45:18.996929 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 14:45:18.998061 2745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.988893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:41627 (local address 127.24.215.130:45191)
0810 14:45:18.989549 (+ 656us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:18.989570 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:18.989702 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:18.990227 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:18.990232 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:18.990258 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:18.990560 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.990570 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.992244 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.992250 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.994558 (+ 2308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.994574 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.996418 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.996425 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.996599 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:18.996632 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:45:18.997437 (+ 805us) client_negotiation.cc:240] Negotiation successful
0810 14:45:18.997609 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":517,"mutex_wait_us":5,"thread_start_us":902,"threads_started":1}
I20260810 14:45:18.999043 2641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8de11e5b0e614b3ba53511af7d36168d" candidate_uuid: "76c1ea19690a4018870aea9314f752d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f913dae965f4b248b25cf6105cc0f1e" is_pre_election: true
I20260810 14:45:18.999634 2641 raft_consensus.cc:2454] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76c1ea19690a4018870aea9314f752d8 in term 0.
I20260810 14:45:19.000360 2392 leader_election.cc:302] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f913dae965f4b248b25cf6105cc0f1e, 76c1ea19690a4018870aea9314f752d8; no voters:
I20260810 14:45:19.000826 2742 raft_consensus.cc:2781] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:45:19.000973 2742 raft_consensus.cc:482] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:45:19.001075 2742 raft_consensus.cc:3037] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:19.001330 2732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.993454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:45831 (local address 127.24.215.129:41597)
0810 14:45:18.993649 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:18.993655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:18.994237 (+ 582us) server_negotiation.cc:407] Connection header received
0810 14:45:18.994446 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:18.994453 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:18.994561 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:18.994663 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:18.995600 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.996798 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.998965 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.000165 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.000287 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.000347 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:19.000929 (+ 582us) server_negotiation.cc:299] Negotiation successful
0810 14:45:19.001077 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 14:45:19.001607 2746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:18.993336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:41597 (local address 127.24.215.130:45831)
0810 14:45:18.994126 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:18.994144 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:18.994295 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:18.994866 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:18.994874 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:18.994909 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:18.995390 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.995407 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:18.997028 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:18.997034 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:18.998803 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:18.998822 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.000611 (+ 1789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.000620 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:19.000754 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.000777 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 14:45:19.000888 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 14:45:19.001034 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":667,"thread_start_us":422,"threads_started":1}
I20260810 14:45:19.002698 2310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8de11e5b0e614b3ba53511af7d36168d" candidate_uuid: "76c1ea19690a4018870aea9314f752d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e892cc64bb7430aa632692d7ad94d5e" is_pre_election: true
I20260810 14:45:19.003242 2310 raft_consensus.cc:2454] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76c1ea19690a4018870aea9314f752d8 in term 0.
I20260810 14:45:19.004402 2742 raft_consensus.cc:504] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } }
I20260810 14:45:19.005100 2742 leader_election.cc:288] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [CANDIDATE]: Term 1 election: Requested vote from peers 2f913dae965f4b248b25cf6105cc0f1e (127.24.215.131:41627), 7e892cc64bb7430aa632692d7ad94d5e (127.24.215.129:41597)
I20260810 14:45:19.005669 2310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8de11e5b0e614b3ba53511af7d36168d" candidate_uuid: "76c1ea19690a4018870aea9314f752d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e892cc64bb7430aa632692d7ad94d5e"
I20260810 14:45:19.005671 2641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8de11e5b0e614b3ba53511af7d36168d" candidate_uuid: "76c1ea19690a4018870aea9314f752d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f913dae965f4b248b25cf6105cc0f1e"
I20260810 14:45:19.005880 2641 raft_consensus.cc:3037] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:19.005892 2310 raft_consensus.cc:3037] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:19.008431 2310 raft_consensus.cc:2454] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76c1ea19690a4018870aea9314f752d8 in term 1.
I20260810 14:45:19.008948 2395 leader_election.cc:302] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76c1ea19690a4018870aea9314f752d8, 7e892cc64bb7430aa632692d7ad94d5e; no voters:
I20260810 14:45:19.009109 2641 raft_consensus.cc:2454] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76c1ea19690a4018870aea9314f752d8 in term 1.
I20260810 14:45:19.009361 2742 raft_consensus.cc:2781] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:45:19.009856 2742 raft_consensus.cc:686] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [term 1 LEADER]: Becoming Leader. State: Replica: 76c1ea19690a4018870aea9314f752d8, State: Running, Role: LEADER
I20260810 14:45:19.010386 2742 consensus_queue.cc:237] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } }
I20260810 14:45:19.015592 2118 catalog_manager.cc:5495] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76c1ea19690a4018870aea9314f752d8 (127.24.215.130). New cstate: current_term: 1 leader_uuid: "76c1ea19690a4018870aea9314f752d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } health_report { overall_health: UNKNOWN } } }
I20260810 14:45:19.033149 2730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.028373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44748 (local address 127.24.215.130:38085)
0810 14:45:19.028562 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:19.028567 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:19.028583 (+ 16us) server_negotiation.cc:407] Connection header received
0810 14:45:19.028669 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:19.028673 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:19.028760 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:19.028899 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:45:19.029198 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.029926 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.031206 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.031605 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.031700 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.032093 (+ 393us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:45:19.032643 (+ 550us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:45:19.032751 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:19.032833 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 14:45:19.032909 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 14:45:19.041270 2641 raft_consensus.cc:1261] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Refusing update from remote peer 76c1ea19690a4018870aea9314f752d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:45:19.041316 2310 raft_consensus.cc:1261] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e [term 1 FOLLOWER]: Refusing update from remote peer 76c1ea19690a4018870aea9314f752d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:45:19.042275 2747 consensus_queue.cc:1035] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:19.042635 2742 consensus_queue.cc:1035] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:19.065472 2753 mvcc.cc:205] Tried to move back new op lower bound from 7316984295582650368 to 7316984295471091712. Current Snapshot: MvccSnapshot[applied={T|T < 7316984295582650368}]
I20260810 14:45:19.073382 2118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:45:19.073662 2118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:45:19.077239 2118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e: Sending DeleteTablet for 3 replicas of tablet 8de11e5b0e614b3ba53511af7d36168d
I20260810 14:45:19.078544 2617 tablet_service.cc:1514] Processing DeleteTablet for tablet 8de11e5b0e614b3ba53511af7d36168d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:19 UTC) from {username='oryx', principal='oryx/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:35790
I20260810 14:45:19.078759 2283 tablet_service.cc:1514] Processing DeleteTablet for tablet 8de11e5b0e614b3ba53511af7d36168d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:19 UTC) from {username='oryx', principal='oryx/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:40950
I20260810 14:45:19.079319 2446 tablet_service.cc:1514] Processing DeleteTablet for tablet 8de11e5b0e614b3ba53511af7d36168d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:19 UTC) from {username='oryx', principal='oryx/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:44744
I20260810 14:45:19.080281 2765 tablet_replica.cc:330] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e: stopping tablet replica
I20260810 14:45:19.080557 2766 tablet_replica.cc:330] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8: stopping tablet replica
I20260810 14:45:19.080499 2118 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34358:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260810 14:45:19.080768 2765 raft_consensus.cc:2229] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e [term 1 FOLLOWER]: Raft consensus shutting down.
W20260810 14:45:19.081210 2118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:45:19.081305 2765 raft_consensus.cc:2258] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:19.081239 2766 raft_consensus.cc:2229] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:45:19.082161 2766 raft_consensus.cc:2258] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:19.083673 2765 ts_tablet_manager.cc:1905] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:19.089797 2283 tablet_service.cc:1467] Processing CreateTablet for tablet f35dc0fdf15d43efab6a2a39438f34fe (DEFAULT_TABLE table=test-table [id=5ad9fa18d7724f579a2280b4264cc7ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:19.090291 2283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f35dc0fdf15d43efab6a2a39438f34fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:19.090327 2764 tablet_replica.cc:330] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e: stopping tablet replica
I20260810 14:45:19.091650 2617 tablet_service.cc:1467] Processing CreateTablet for tablet f35dc0fdf15d43efab6a2a39438f34fe (DEFAULT_TABLE table=test-table [id=5ad9fa18d7724f579a2280b4264cc7ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:19.092145 2617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f35dc0fdf15d43efab6a2a39438f34fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:19.092191 2764 raft_consensus.cc:2229] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:45:19.092784 2764 raft_consensus.cc:2258] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:19.093422 2766 ts_tablet_manager.cc:1905] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:19.094738 2446 tablet_service.cc:1467] Processing CreateTablet for tablet f35dc0fdf15d43efab6a2a39438f34fe (DEFAULT_TABLE table=test-table [id=5ad9fa18d7724f579a2280b4264cc7ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:19.095402 2446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f35dc0fdf15d43efab6a2a39438f34fe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:19.095628 2765 ts_tablet_manager.cc:1918] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:45:19.095805 2765 log.cc:1196] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/8de11e5b0e614b3ba53511af7d36168d
I20260810 14:45:19.096379 2765 ts_tablet_manager.cc:1939] T 8de11e5b0e614b3ba53511af7d36168d P 7e892cc64bb7430aa632692d7ad94d5e: Deleting consensus metadata
I20260810 14:45:19.098106 2764 ts_tablet_manager.cc:1905] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:19.098510 2103 catalog_manager.cc:4841] TS 7e892cc64bb7430aa632692d7ad94d5e (127.24.215.129:41597): tablet 8de11e5b0e614b3ba53511af7d36168d (table test-table [id=8a0a4a2d23444f13853a1933bcc82702]) successfully deleted
I20260810 14:45:19.100951 2734 tablet_bootstrap.cc:492] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e: Bootstrap starting.
I20260810 14:45:19.101982 2733 tablet_bootstrap.cc:492] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e: Bootstrap starting.
I20260810 14:45:19.102679 2734 tablet_bootstrap.cc:654] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:19.103741 2733 tablet_bootstrap.cc:654] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:19.107842 2737 tablet_bootstrap.cc:492] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8: Bootstrap starting.
I20260810 14:45:19.109834 2737 tablet_bootstrap.cc:654] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:19.115684 2764 ts_tablet_manager.cc:1918] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:45:19.115922 2764 log.cc:1196] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/8de11e5b0e614b3ba53511af7d36168d
I20260810 14:45:19.116686 2764 ts_tablet_manager.cc:1939] T 8de11e5b0e614b3ba53511af7d36168d P 2f913dae965f4b248b25cf6105cc0f1e: Deleting consensus metadata
I20260810 14:45:19.119179 2101 catalog_manager.cc:4841] TS 2f913dae965f4b248b25cf6105cc0f1e (127.24.215.131:41627): tablet 8de11e5b0e614b3ba53511af7d36168d (table test-table [id=8a0a4a2d23444f13853a1933bcc82702]) successfully deleted
I20260810 14:45:19.121933 2766 ts_tablet_manager.cc:1918] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:45:19.122170 2766 log.cc:1196] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/8de11e5b0e614b3ba53511af7d36168d
I20260810 14:45:19.122174 2734 tablet_bootstrap.cc:492] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e: No bootstrap required, opened a new log
I20260810 14:45:19.122350 2734 ts_tablet_manager.cc:1397] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e: Time spent bootstrapping tablet: real 0.021s user 0.001s sys 0.014s
I20260810 14:45:19.122800 2766 ts_tablet_manager.cc:1939] T 8de11e5b0e614b3ba53511af7d36168d P 76c1ea19690a4018870aea9314f752d8: Deleting consensus metadata
I20260810 14:45:19.123412 2733 tablet_bootstrap.cc:492] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e: No bootstrap required, opened a new log
I20260810 14:45:19.123308 2734 raft_consensus.cc:348] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.123560 2733 ts_tablet_manager.cc:1397] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e: Time spent bootstrapping tablet: real 0.022s user 0.007s sys 0.004s
I20260810 14:45:19.123566 2734 raft_consensus.cc:374] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:19.123661 2734 raft_consensus.cc:729] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e892cc64bb7430aa632692d7ad94d5e, State: Initialized, Role: FOLLOWER
I20260810 14:45:19.123965 2734 consensus_queue.cc:260] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.124496 2733 raft_consensus.cc:348] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.124723 2733 raft_consensus.cc:374] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:19.124877 2104 catalog_manager.cc:4841] TS 76c1ea19690a4018870aea9314f752d8 (127.24.215.130:38085): tablet 8de11e5b0e614b3ba53511af7d36168d (table test-table [id=8a0a4a2d23444f13853a1933bcc82702]) successfully deleted
I20260810 14:45:19.125223 2734 ts_tablet_manager.cc:1428] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260810 14:45:19.124827 2733 raft_consensus.cc:729] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f913dae965f4b248b25cf6105cc0f1e, State: Initialized, Role: FOLLOWER
I20260810 14:45:19.125551 2733 consensus_queue.cc:260] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.126259 2733 ts_tablet_manager.cc:1428] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260810 14:45:19.127895 2737 tablet_bootstrap.cc:492] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8: No bootstrap required, opened a new log
I20260810 14:45:19.128062 2737 ts_tablet_manager.cc:1397] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8: Time spent bootstrapping tablet: real 0.020s user 0.006s sys 0.001s
I20260810 14:45:19.129002 2737 raft_consensus.cc:348] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.129258 2737 raft_consensus.cc:374] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:19.129390 2737 raft_consensus.cc:729] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c1ea19690a4018870aea9314f752d8, State: Initialized, Role: FOLLOWER
I20260810 14:45:19.129673 2737 consensus_queue.cc:260] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.130396 2737 ts_tablet_manager.cc:1428] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260810 14:45:19.147367 2743 raft_consensus.cc:482] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:45:19.147678 2743 raft_consensus.cc:504] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.149724 2743 leader_election.cc:288] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76c1ea19690a4018870aea9314f752d8 (127.24.215.130:38085), 2f913dae965f4b248b25cf6105cc0f1e (127.24.215.131:41627)
I20260810 14:45:19.158716 2730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.150368 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:41647 (local address 127.24.215.130:38085)
0810 14:45:19.150630 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:19.150641 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:19.151464 (+ 823us) server_negotiation.cc:407] Connection header received
0810 14:45:19.151778 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:19.151784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:19.151873 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:19.151992 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:19.152756 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.154051 (+ 1295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.156789 (+ 2738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.157643 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.157847 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.157939 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:19.158236 (+ 297us) server_negotiation.cc:299] Negotiation successful
0810 14:45:19.158373 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 14:45:19.158862 2771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.149916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:38085 (local address 127.24.215.129:41647)
0810 14:45:19.151315 (+ 1399us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:19.151341 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:19.151594 (+ 253us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:19.152029 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:19.152036 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:19.152073 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:19.152546 (+ 473us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:19.152559 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.154207 (+ 1648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.154213 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:19.156551 (+ 2338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:19.156571 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.157875 (+ 1304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.157882 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:19.158064 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.158090 (+ 26us) client_negotiation.cc:769] Sending connection context
0810 14:45:19.158207 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 14:45:19.158354 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1214,"mutex_wait_us":26,"thread_start_us":1004,"threads_started":1}
I20260810 14:45:19.159490 2727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.150696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:46713 (local address 127.24.215.131:41627)
0810 14:45:19.150882 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:19.150891 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:19.151756 (+ 865us) server_negotiation.cc:407] Connection header received
0810 14:45:19.151853 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:19.151859 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:19.151952 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:19.152051 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:19.152970 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.153739 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.156722 (+ 2983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.158084 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.158253 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.158345 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:19.159001 (+ 656us) server_negotiation.cc:299] Negotiation successful
0810 14:45:19.159192 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:45:19.160323 2772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.150145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:41627 (local address 127.24.215.129:46713)
0810 14:45:19.151592 (+ 1447us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:19.151610 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:19.151726 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:19.152243 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:19.152248 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:19.152270 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:19.152789 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:19.152802 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.153905 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.153911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:19.156551 (+ 2640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:19.156582 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.158541 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.158550 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:19.158761 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.158807 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 14:45:19.159738 (+ 931us) client_negotiation.cc:240] Negotiation successful
0810 14:45:19.159899 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1332,"mutex_wait_us":37,"thread_start_us":1149,"threads_started":1}
I20260810 14:45:19.160045 2471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f35dc0fdf15d43efab6a2a39438f34fe" candidate_uuid: "7e892cc64bb7430aa632692d7ad94d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76c1ea19690a4018870aea9314f752d8" is_pre_election: true
I20260810 14:45:19.160652 2471 raft_consensus.cc:2454] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e892cc64bb7430aa632692d7ad94d5e in term 0.
I20260810 14:45:19.160946 2641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f35dc0fdf15d43efab6a2a39438f34fe" candidate_uuid: "7e892cc64bb7430aa632692d7ad94d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f913dae965f4b248b25cf6105cc0f1e" is_pre_election: true
I20260810 14:45:19.161239 2641 raft_consensus.cc:2454] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e892cc64bb7430aa632692d7ad94d5e in term 0.
I20260810 14:45:19.161430 2226 leader_election.cc:302] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76c1ea19690a4018870aea9314f752d8, 7e892cc64bb7430aa632692d7ad94d5e; no voters:
I20260810 14:45:19.161904 2743 raft_consensus.cc:2781] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:45:19.162061 2743 raft_consensus.cc:482] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:45:19.162190 2743 raft_consensus.cc:3037] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:19.164577 2743 raft_consensus.cc:504] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.165252 2743 leader_election.cc:288] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [CANDIDATE]: Term 1 election: Requested vote from peers 76c1ea19690a4018870aea9314f752d8 (127.24.215.130:38085), 2f913dae965f4b248b25cf6105cc0f1e (127.24.215.131:41627)
I20260810 14:45:19.165763 2641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f35dc0fdf15d43efab6a2a39438f34fe" candidate_uuid: "7e892cc64bb7430aa632692d7ad94d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f913dae965f4b248b25cf6105cc0f1e"
I20260810 14:45:19.165799 2471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f35dc0fdf15d43efab6a2a39438f34fe" candidate_uuid: "7e892cc64bb7430aa632692d7ad94d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76c1ea19690a4018870aea9314f752d8"
I20260810 14:45:19.165990 2641 raft_consensus.cc:3037] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:19.166040 2471 raft_consensus.cc:3037] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:19.168305 2641 raft_consensus.cc:2454] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e892cc64bb7430aa632692d7ad94d5e in term 1.
I20260810 14:45:19.168792 2223 leader_election.cc:302] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f913dae965f4b248b25cf6105cc0f1e, 7e892cc64bb7430aa632692d7ad94d5e; no voters:
I20260810 14:45:19.169100 2743 raft_consensus.cc:2781] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:45:19.169530 2471 raft_consensus.cc:2454] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e892cc64bb7430aa632692d7ad94d5e in term 1.
I20260810 14:45:19.169658 2743 raft_consensus.cc:686] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [term 1 LEADER]: Becoming Leader. State: Replica: 7e892cc64bb7430aa632692d7ad94d5e, State: Running, Role: LEADER
I20260810 14:45:19.170149 2743 consensus_queue.cc:237] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.174125 2119 catalog_manager.cc:5495] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e892cc64bb7430aa632692d7ad94d5e (127.24.215.129). New cstate: current_term: 1 leader_uuid: "7e892cc64bb7430aa632692d7ad94d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } health_report { overall_health: UNKNOWN } } }
Aug 10 14:45:19 dist-test-slave-vkjp krb5kdc[2067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373116, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.190@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM
I20260810 14:45:19.245025 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.229792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:43325 (local address 127.0.0.1:34368)
0810 14:45:19.230524 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:19.230541 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:19.230666 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:19.231060 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:19.231064 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:19.231788 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:19.232044 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:19.232054 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.233500 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.233504 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:19.234622 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:19.234635 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.235672 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.235677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:19.235794 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.236475 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:19.236492 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:19.238806 (+ 2314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:19.242602 (+ 3796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:19.242612 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:19.242618 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:19.243050 (+ 432us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:19.243479 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:19.243482 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:19.243484 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:19.243615 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:19.244212 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:19.244218 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:19.244370 (+ 152us) client_negotiation.cc:769] Sending connection context
0810 14:45:19.244508 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 14:45:19.244703 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"thread_start_us":372,"threads_started":1}
I20260810 14:45:19.245290 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.229776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34368 (local address 127.24.215.190:43325)
0810 14:45:19.230061 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:19.230069 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:19.230630 (+ 561us) server_negotiation.cc:407] Connection header received
0810 14:45:19.230791 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:19.230798 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:19.230887 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:19.231015 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:19.232203 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.233367 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.234767 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.235495 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.235632 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.239011 (+ 3379us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:19.239019 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:19.239024 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:19.239062 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:19.242475 (+ 3413us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:19.243196 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:19.243202 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:19.243206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:19.243364 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:19.243717 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:19.243722 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:19.243726 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:19.244095 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:19.244247 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:19.244635 (+ 388us) server_negotiation.cc:299] Negotiation successful
0810 14:45:19.244868 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260810 14:45:19.250865 2119 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:34368:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260810 14:45:19.252007 2119 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:45:19.259145 2283 tablet_service.cc:1467] Processing CreateTablet for tablet 1e00523214c942a6802729d02908a5a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=32d06a44035a4eea99f6fb66b5954af8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:45:19.259158 2446 tablet_service.cc:1467] Processing CreateTablet for tablet 1e00523214c942a6802729d02908a5a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=32d06a44035a4eea99f6fb66b5954af8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:45:19.259316 2617 tablet_service.cc:1467] Processing CreateTablet for tablet 1e00523214c942a6802729d02908a5a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=32d06a44035a4eea99f6fb66b5954af8]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260810 14:45:19.259773 2446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e00523214c942a6802729d02908a5a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:19.259778 2283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e00523214c942a6802729d02908a5a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:19.259783 2617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e00523214c942a6802729d02908a5a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:19.267071 2733 tablet_bootstrap.cc:492] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e: Bootstrap starting.
I20260810 14:45:19.267257 2737 tablet_bootstrap.cc:492] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8: Bootstrap starting.
I20260810 14:45:19.268878 2733 tablet_bootstrap.cc:654] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:19.268945 2737 tablet_bootstrap.cc:654] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:19.271265 2734 tablet_bootstrap.cc:492] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e: Bootstrap starting.
I20260810 14:45:19.272985 2734 tablet_bootstrap.cc:654] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:19.273463 2733 tablet_bootstrap.cc:492] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e: No bootstrap required, opened a new log
I20260810 14:45:19.273639 2733 ts_tablet_manager.cc:1397] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.003s
I20260810 14:45:19.274477 2733 raft_consensus.cc:348] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.274746 2733 raft_consensus.cc:374] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:19.274796 2737 tablet_bootstrap.cc:492] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8: No bootstrap required, opened a new log
I20260810 14:45:19.274844 2733 raft_consensus.cc:729] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f913dae965f4b248b25cf6105cc0f1e, State: Initialized, Role: FOLLOWER
I20260810 14:45:19.274961 2737 ts_tablet_manager.cc:1397] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.000s
I20260810 14:45:19.275180 2733 consensus_queue.cc:260] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.275545 2734 tablet_bootstrap.cc:492] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e: No bootstrap required, opened a new log
I20260810 14:45:19.275758 2734 ts_tablet_manager.cc:1397] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260810 14:45:19.275645 2744 tablet_replica.cc:439] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } }
I20260810 14:45:19.275738 2737 raft_consensus.cc:348] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.275957 2744 tablet_replica.cc:442] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:45:19.276003 2737 raft_consensus.cc:374] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:19.276104 2737 raft_consensus.cc:729] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c1ea19690a4018870aea9314f752d8, State: Initialized, Role: FOLLOWER
I20260810 14:45:19.276335 2737 consensus_queue.cc:260] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.276947 2737 ts_tablet_manager.cc:1428] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:45:19.276858 2734 raft_consensus.cc:348] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.276913 2742 tablet_replica.cc:439] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } }
I20260810 14:45:19.277220 2742 tablet_replica.cc:442] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:45:19.277243 2734 raft_consensus.cc:374] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:19.277324 2734 raft_consensus.cc:729] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e892cc64bb7430aa632692d7ad94d5e, State: Initialized, Role: FOLLOWER
I20260810 14:45:19.277576 2734 consensus_queue.cc:260] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.278018 2743 tablet_replica.cc:439] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } }
I20260810 14:45:19.278216 2734 ts_tablet_manager.cc:1428] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260810 14:45:19.278312 2743 tablet_replica.cc:442] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:45:19.279774 2733 ts_tablet_manager.cc:1428] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e: Time spent starting tablet: real 0.006s user 0.003s sys 0.000s
I20260810 14:45:19.326910 2743 raft_consensus.cc:482] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:45:19.327169 2743 raft_consensus.cc:504] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.327788 2743 leader_election.cc:288] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76c1ea19690a4018870aea9314f752d8 (127.24.215.130:38085), 2f913dae965f4b248b25cf6105cc0f1e (127.24.215.131:41627)
I20260810 14:45:19.328306 2471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e00523214c942a6802729d02908a5a3" candidate_uuid: "7e892cc64bb7430aa632692d7ad94d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76c1ea19690a4018870aea9314f752d8" is_pre_election: true
I20260810 14:45:19.328361 2641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e00523214c942a6802729d02908a5a3" candidate_uuid: "7e892cc64bb7430aa632692d7ad94d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f913dae965f4b248b25cf6105cc0f1e" is_pre_election: true
I20260810 14:45:19.328611 2471 raft_consensus.cc:2454] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e892cc64bb7430aa632692d7ad94d5e in term 0.
I20260810 14:45:19.328634 2641 raft_consensus.cc:2454] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e892cc64bb7430aa632692d7ad94d5e in term 0.
I20260810 14:45:19.329064 2226 leader_election.cc:302] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76c1ea19690a4018870aea9314f752d8, 7e892cc64bb7430aa632692d7ad94d5e; no voters:
I20260810 14:45:19.329376 2743 raft_consensus.cc:2781] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:45:19.329489 2743 raft_consensus.cc:482] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:45:19.329593 2743 raft_consensus.cc:3037] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:19.332656 2743 raft_consensus.cc:504] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.333333 2743 leader_election.cc:288] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [CANDIDATE]: Term 1 election: Requested vote from peers 76c1ea19690a4018870aea9314f752d8 (127.24.215.130:38085), 2f913dae965f4b248b25cf6105cc0f1e (127.24.215.131:41627)
I20260810 14:45:19.333762 2641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e00523214c942a6802729d02908a5a3" candidate_uuid: "7e892cc64bb7430aa632692d7ad94d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f913dae965f4b248b25cf6105cc0f1e"
I20260810 14:45:19.333766 2471 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e00523214c942a6802729d02908a5a3" candidate_uuid: "7e892cc64bb7430aa632692d7ad94d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76c1ea19690a4018870aea9314f752d8"
I20260810 14:45:19.334020 2471 raft_consensus.cc:3037] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:19.334022 2641 raft_consensus.cc:3037] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:19.336162 2641 raft_consensus.cc:2454] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e892cc64bb7430aa632692d7ad94d5e in term 1.
I20260810 14:45:19.336587 2223 leader_election.cc:302] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f913dae965f4b248b25cf6105cc0f1e, 7e892cc64bb7430aa632692d7ad94d5e; no voters:
I20260810 14:45:19.336848 2743 raft_consensus.cc:2781] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:45:19.337265 2743 raft_consensus.cc:686] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [term 1 LEADER]: Becoming Leader. State: Replica: 7e892cc64bb7430aa632692d7ad94d5e, State: Running, Role: LEADER
I20260810 14:45:19.337309 2471 raft_consensus.cc:2454] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e892cc64bb7430aa632692d7ad94d5e in term 1.
I20260810 14:45:19.337636 2743 consensus_queue.cc:237] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.340370 2774 tablet_replica.cc:439] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e: TxnStatusTablet state changed. Reason: New leader 7e892cc64bb7430aa632692d7ad94d5e. Latest consensus state: current_term: 1 leader_uuid: "7e892cc64bb7430aa632692d7ad94d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } }
I20260810 14:45:19.340750 2774 tablet_replica.cc:442] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:45:19.340703 2118 catalog_manager.cc:5495] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e892cc64bb7430aa632692d7ad94d5e (127.24.215.129). New cstate: current_term: 1 leader_uuid: "7e892cc64bb7430aa632692d7ad94d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } health_report { overall_health: UNKNOWN } } }
I20260810 14:45:19.341559 2788 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260810 14:45:19.341813 2788 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260810 14:45:19.370777 2781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.365191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:41597 (local address 127.0.0.1:40956)
0810 14:45:19.365411 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:19.365432 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:19.365577 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:19.366055 (+ 478us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:19.366059 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:19.366071 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:45:19.366268 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:19.366277 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.367351 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.367358 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:19.368380 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:19.368393 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.369166 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.369172 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:19.369339 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.369391 (+ 52us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:45:19.370213 (+ 822us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:45:19.370247 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 14:45:19.370333 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 14:45:19.370489 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 14:45:19.370788 2732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.365506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40956 (local address 127.24.215.129:41597)
0810 14:45:19.365688 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:19.365696 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:19.365714 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:45:19.365801 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:19.365805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:19.365889 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:19.366036 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:45:19.366402 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.367228 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.368536 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.369011 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.369149 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.369598 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:45:19.370099 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:45:19.370263 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:19.370356 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 14:45:19.370436 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:45:19.374642 2641 raft_consensus.cc:1261] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Refusing update from remote peer 7e892cc64bb7430aa632692d7ad94d5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:45:19.375335 2471 raft_consensus.cc:1261] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8 [term 1 FOLLOWER]: Refusing update from remote peer 7e892cc64bb7430aa632692d7ad94d5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:45:19.376283 2743 consensus_queue.cc:1035] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:19.376904 2743 consensus_queue.cc:1035] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:19.391069 2742 tablet_replica.cc:439] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8: TxnStatusTablet state changed. Reason: New leader 7e892cc64bb7430aa632692d7ad94d5e. Latest consensus state: current_term: 1 leader_uuid: "7e892cc64bb7430aa632692d7ad94d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } }
I20260810 14:45:19.391444 2742 tablet_replica.cc:442] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:45:19.391479 2744 tablet_replica.cc:439] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e: TxnStatusTablet state changed. Reason: New leader 7e892cc64bb7430aa632692d7ad94d5e. Latest consensus state: current_term: 1 leader_uuid: "7e892cc64bb7430aa632692d7ad94d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } }
I20260810 14:45:19.391743 2744 tablet_replica.cc:442] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:45:19.394477 2774 tablet_replica.cc:439] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7e892cc64bb7430aa632692d7ad94d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } }
I20260810 14:45:19.394724 2774 tablet_replica.cc:442] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:45:19.394912 2751 mvcc.cc:205] Tried to move back new op lower bound from 7316984296951341056 to 7316984296809881600. Current Snapshot: MvccSnapshot[applied={T|T < 7316984296951341056 or (T in {7316984296951341056})}]
I20260810 14:45:19.396648 2742 tablet_replica.cc:439] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7e892cc64bb7430aa632692d7ad94d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } }
I20260810 14:45:19.396921 2742 tablet_replica.cc:442] T 1e00523214c942a6802729d02908a5a3 P 76c1ea19690a4018870aea9314f752d8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:45:19.400944 2743 tablet_replica.cc:439] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7e892cc64bb7430aa632692d7ad94d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } }
I20260810 14:45:19.401328 2743 tablet_replica.cc:442] T 1e00523214c942a6802729d02908a5a3 P 7e892cc64bb7430aa632692d7ad94d5e: This TxnStatusTablet replica's current role is: LEADER
I20260810 14:45:19.403120 2744 tablet_replica.cc:439] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7e892cc64bb7430aa632692d7ad94d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } } }
I20260810 14:45:19.403360 2744 tablet_replica.cc:442] T 1e00523214c942a6802729d02908a5a3 P 2f913dae965f4b248b25cf6105cc0f1e: This TxnStatusTablet replica's current role is: FOLLOWER
I20260810 14:45:19.409183 2732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.402567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40968 (local address 127.24.215.129:41597)
0810 14:45:19.402852 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:19.402857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:19.402874 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:45:19.402932 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:19.402937 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:19.403018 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:19.403134 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:45:19.404280 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.405454 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.406886 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.407635 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.407771 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.408385 (+ 614us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:45:19.408499 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:45:19.408607 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:19.408760 (+ 153us) server_negotiation.cc:299] Negotiation successful
0810 14:45:19.408836 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260810 14:45:19.425874 2732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.419330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:35339 (local address 127.24.215.129:41597)
0810 14:45:19.419561 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:19.419569 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:19.420328 (+ 759us) server_negotiation.cc:407] Connection header received
0810 14:45:19.420550 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:19.420555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:19.420623 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:19.420704 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:45:19.421216 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.421986 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.423897 (+ 1911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.424515 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.424596 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.425079 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:45:19.425203 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:45:19.425296 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:19.425401 (+ 105us) server_negotiation.cc:299] Negotiation successful
0810 14:45:19.425508 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":53}
I20260810 14:45:19.425997 2795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.419430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:41597 (local address 127.24.215.129:35339)
0810 14:45:19.420237 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:19.420254 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:19.420403 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:19.420729 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:19.420733 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:19.420744 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0810 14:45:19.421075 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:19.421084 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.422126 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.422131 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:19.423710 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:19.423730 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.424749 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.424753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:19.424850 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.424897 (+ 47us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0810 14:45:19.425297 (+ 400us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0810 14:45:19.425309 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 14:45:19.425386 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 14:45:19.425564 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":637,"mutex_wait_us":24,"thread_start_us":433,"threads_started":1}
I20260810 14:45:19.439996 2471 raft_consensus.cc:1261] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8 [term 1 FOLLOWER]: Refusing update from remote peer 7e892cc64bb7430aa632692d7ad94d5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:45:19.440698 2774 consensus_queue.cc:1035] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:19.440889 2641 raft_consensus.cc:1261] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Refusing update from remote peer 7e892cc64bb7430aa632692d7ad94d5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:45:19.441709 2743 consensus_queue.cc:1035] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:19.534284 2118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:45:19.534554 2118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:34358:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:45:19.537806 2118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e: Sending DeleteTablet for 3 replicas of tablet f35dc0fdf15d43efab6a2a39438f34fe
I20260810 14:45:19.538972 2446 tablet_service.cc:1514] Processing DeleteTablet for tablet f35dc0fdf15d43efab6a2a39438f34fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:19 UTC) from {username='oryx', principal='oryx/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:44744
I20260810 14:45:19.539436 2766 tablet_replica.cc:330] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8: stopping tablet replica
I20260810 14:45:19.539667 2766 raft_consensus.cc:2229] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:45:19.539965 2766 raft_consensus.cc:2258] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:19.540845 2617 tablet_service.cc:1514] Processing DeleteTablet for tablet f35dc0fdf15d43efab6a2a39438f34fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:19 UTC) from {username='oryx', principal='oryx/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:35790
I20260810 14:45:19.541533 2764 tablet_replica.cc:330] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e: stopping tablet replica
I20260810 14:45:19.541769 2764 raft_consensus.cc:2229] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:45:19.541967 2764 raft_consensus.cc:2258] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:19.542128 2766 ts_tablet_manager.cc:1905] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:19.542945 2283 tablet_service.cc:1514] Processing DeleteTablet for tablet f35dc0fdf15d43efab6a2a39438f34fe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:19 UTC) from {username='oryx', principal='oryx/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:40950
I20260810 14:45:19.543246 2765 tablet_replica.cc:330] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e: stopping tablet replica
I20260810 14:45:19.543500 2765 raft_consensus.cc:2229] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:45:19.544023 2765 raft_consensus.cc:2258] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:19.545256 2765 ts_tablet_manager.cc:1905] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:19.555068 2765 ts_tablet_manager.cc:1918] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:45:19.555315 2765 log.cc:1196] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/f35dc0fdf15d43efab6a2a39438f34fe
I20260810 14:45:19.556085 2765 ts_tablet_manager.cc:1939] T f35dc0fdf15d43efab6a2a39438f34fe P 7e892cc64bb7430aa632692d7ad94d5e: Deleting consensus metadata
I20260810 14:45:19.557586 2103 catalog_manager.cc:4841] TS 7e892cc64bb7430aa632692d7ad94d5e (127.24.215.129:41597): tablet f35dc0fdf15d43efab6a2a39438f34fe (table test-table [id=5ad9fa18d7724f579a2280b4264cc7ea]) successfully deleted
I20260810 14:45:19.562544 2766 ts_tablet_manager.cc:1918] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:45:19.562795 2766 log.cc:1196] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/f35dc0fdf15d43efab6a2a39438f34fe
I20260810 14:45:19.563482 2766 ts_tablet_manager.cc:1939] T f35dc0fdf15d43efab6a2a39438f34fe P 76c1ea19690a4018870aea9314f752d8: Deleting consensus metadata
I20260810 14:45:19.564291 2764 ts_tablet_manager.cc:1905] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:19.564790 2104 catalog_manager.cc:4841] TS 76c1ea19690a4018870aea9314f752d8 (127.24.215.130:38085): tablet f35dc0fdf15d43efab6a2a39438f34fe (table test-table [id=5ad9fa18d7724f579a2280b4264cc7ea]) successfully deleted
I20260810 14:45:19.571684 2764 ts_tablet_manager.cc:1918] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:45:19.571923 2764 log.cc:1196] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/f35dc0fdf15d43efab6a2a39438f34fe
I20260810 14:45:19.572453 2764 ts_tablet_manager.cc:1939] T f35dc0fdf15d43efab6a2a39438f34fe P 2f913dae965f4b248b25cf6105cc0f1e: Deleting consensus metadata
I20260810 14:45:19.574060 2101 catalog_manager.cc:4841] TS 2f913dae965f4b248b25cf6105cc0f1e (127.24.215.131:41627): tablet f35dc0fdf15d43efab6a2a39438f34fe (table test-table [id=5ad9fa18d7724f579a2280b4264cc7ea]) successfully deleted
Aug 10 14:45:19 dist-test-slave-vkjp krb5kdc[2067](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 10 14:45:19 dist-test-slave-vkjp krb5kdc[2067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:19 dist-test-slave-vkjp krb5kdc[2067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 14:45:19.638252 2401 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:19.624662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34380 (local address 127.24.215.190:43325)
0810 14:45:19.624985 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:19.624993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:19.625366 (+ 373us) server_negotiation.cc:407] Connection header received
0810 14:45:19.625587 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:19.625593 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:19.625684 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:19.625795 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:19.627076 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.627952 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.629256 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.629704 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.629857 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.637987 (+ 8130us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34380: BlockingRecv error: recv got EOF from 127.0.0.1:34380 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 10 14:45:19 dist-test-slave-vkjp krb5kdc[2067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM
I20260810 14:45:19.706902 2401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.674876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34396 (local address 127.24.215.190:43325)
0810 14:45:19.675100 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:19.675107 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:19.689254 (+ 14147us) server_negotiation.cc:407] Connection header received
0810 14:45:19.689567 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:19.689573 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:19.689661 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:19.689733 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:19.691575 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.692716 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.694554 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.695250 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.695357 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.700558 (+ 5201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:19.700565 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:19.700569 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:19.700625 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:19.703962 (+ 3337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:19.704775 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:19.704781 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:19.704785 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:19.704945 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:19.705368 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:19.705373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:19.705376 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:19.705741 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:19.705982 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:19.706300 (+ 318us) server_negotiation.cc:299] Negotiation successful
0810 14:45:19.706552 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 14:45:19.710620 2118 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:45:19.711402 2118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:45:19.718899 2446 tablet_service.cc:1467] Processing CreateTablet for tablet f72db654cbdb4db1aa344587bc5f0589 (DEFAULT_TABLE table=test-table [id=cae62b51af164067bc8836514435efa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:19.719393 2283 tablet_service.cc:1467] Processing CreateTablet for tablet f72db654cbdb4db1aa344587bc5f0589 (DEFAULT_TABLE table=test-table [id=cae62b51af164067bc8836514435efa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:19.719570 2446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f72db654cbdb4db1aa344587bc5f0589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:19.719830 2283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f72db654cbdb4db1aa344587bc5f0589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:19.726845 2617 tablet_service.cc:1467] Processing CreateTablet for tablet f72db654cbdb4db1aa344587bc5f0589 (DEFAULT_TABLE table=test-table [id=cae62b51af164067bc8836514435efa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:19.727113 2737 tablet_bootstrap.cc:492] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8: Bootstrap starting.
I20260810 14:45:19.727324 2617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f72db654cbdb4db1aa344587bc5f0589. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:19.728827 2737 tablet_bootstrap.cc:654] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:19.734755 2734 tablet_bootstrap.cc:492] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e: Bootstrap starting.
I20260810 14:45:19.737040 2734 tablet_bootstrap.cc:654] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:19.740319 2733 tablet_bootstrap.cc:492] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e: Bootstrap starting.
I20260810 14:45:19.742177 2733 tablet_bootstrap.cc:654] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:19.743351 2737 tablet_bootstrap.cc:492] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8: No bootstrap required, opened a new log
I20260810 14:45:19.743614 2737 ts_tablet_manager.cc:1397] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.009s
I20260810 14:45:19.744632 2737 raft_consensus.cc:348] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.745174 2737 raft_consensus.cc:374] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:19.745333 2737 raft_consensus.cc:729] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c1ea19690a4018870aea9314f752d8, State: Initialized, Role: FOLLOWER
I20260810 14:45:19.745703 2737 consensus_queue.cc:260] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.750650 2737 ts_tablet_manager.cc:1428] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8: Time spent starting tablet: real 0.007s user 0.002s sys 0.000s
I20260810 14:45:19.760967 2734 tablet_bootstrap.cc:492] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e: No bootstrap required, opened a new log
I20260810 14:45:19.761175 2734 ts_tablet_manager.cc:1397] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e: Time spent bootstrapping tablet: real 0.027s user 0.006s sys 0.008s
I20260810 14:45:19.762056 2733 tablet_bootstrap.cc:492] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e: No bootstrap required, opened a new log
I20260810 14:45:19.762238 2733 ts_tablet_manager.cc:1397] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.006s
I20260810 14:45:19.762110 2734 raft_consensus.cc:348] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.762398 2734 raft_consensus.cc:374] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:19.762516 2734 raft_consensus.cc:729] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e892cc64bb7430aa632692d7ad94d5e, State: Initialized, Role: FOLLOWER
I20260810 14:45:19.762750 2734 consensus_queue.cc:260] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.763047 2733 raft_consensus.cc:348] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.763335 2733 raft_consensus.cc:374] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:19.763401 2734 ts_tablet_manager.cc:1428] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:45:19.763458 2733 raft_consensus.cc:729] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f913dae965f4b248b25cf6105cc0f1e, State: Initialized, Role: FOLLOWER
I20260810 14:45:19.763705 2733 consensus_queue.cc:260] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.764439 2733 ts_tablet_manager.cc:1428] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260810 14:45:19.964269 2744 raft_consensus.cc:482] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:45:19.964500 2744 raft_consensus.cc:504] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.965977 2744 leader_election.cc:288] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e892cc64bb7430aa632692d7ad94d5e (127.24.215.129:41597), 76c1ea19690a4018870aea9314f752d8 (127.24.215.130:38085)
I20260810 14:45:19.981297 2742 raft_consensus.cc:482] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:45:19.981539 2742 raft_consensus.cc:504] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.982308 2742 leader_election.cc:288] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e892cc64bb7430aa632692d7ad94d5e (127.24.215.129:41597), 2f913dae965f4b248b25cf6105cc0f1e (127.24.215.131:41627)
I20260810 14:45:19.983151 2310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f72db654cbdb4db1aa344587bc5f0589" candidate_uuid: "76c1ea19690a4018870aea9314f752d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e892cc64bb7430aa632692d7ad94d5e" is_pre_election: true
I20260810 14:45:19.983481 2310 raft_consensus.cc:2454] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76c1ea19690a4018870aea9314f752d8 in term 0.
I20260810 14:45:19.983959 2395 leader_election.cc:302] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76c1ea19690a4018870aea9314f752d8, 7e892cc64bb7430aa632692d7ad94d5e; no voters:
I20260810 14:45:19.984241 2742 raft_consensus.cc:2781] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:45:19.984359 2742 raft_consensus.cc:482] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:45:19.984478 2742 raft_consensus.cc:3037] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:19.987589 2822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.966240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:38085 (local address 127.24.215.131:58085)
0810 14:45:19.979023 (+ 12783us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:19.979051 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:19.979241 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:19.980216 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:19.980223 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:19.980251 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:19.980585 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:19.980597 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.982147 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.982153 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:19.984395 (+ 2242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:19.984411 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.986717 (+ 2306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.986723 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:19.986900 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.986933 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:45:19.987029 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 14:45:19.987187 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12583,"thread_start_us":13522,"threads_started":1}
I20260810 14:45:19.988286 2823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.966372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:58085 (local address 127.24.215.130:38085)
0810 14:45:19.979710 (+ 13338us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:19.979716 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:19.979737 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:45:19.979811 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:19.979816 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:19.979901 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:19.980000 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:19.980801 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.981961 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.984624 (+ 2663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.986294 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.986487 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.986569 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:19.987814 (+ 1245us) server_negotiation.cc:299] Negotiation successful
0810 14:45:19.987968 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13215,"thread_start_us":13539,"threads_started":1}
I20260810 14:45:19.988147 2742 raft_consensus.cc:504] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.988982 2821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.966302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:41597 (local address 127.24.215.131:58683)
0810 14:45:19.979576 (+ 13274us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:19.979593 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:19.979696 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:19.980800 (+ 1104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:19.980804 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:19.980828 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:19.981281 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:19.981296 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.983108 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.983116 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:19.985675 (+ 2559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:19.985694 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.987977 (+ 2283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.987986 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:19.988184 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.988219 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 14:45:19.988309 (+ 90us) client_negotiation.cc:240] Negotiation successful
0810 14:45:19.988601 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13146,"thread_start_us":12359,"threads_started":1}
I20260810 14:45:19.989356 2310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f72db654cbdb4db1aa344587bc5f0589" candidate_uuid: "76c1ea19690a4018870aea9314f752d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e892cc64bb7430aa632692d7ad94d5e"
I20260810 14:45:19.989590 2310 raft_consensus.cc:3037] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:19.990118 2824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:19.966770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:58683 (local address 127.24.215.129:41597)
0810 14:45:19.980479 (+ 13709us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:19.980488 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:19.980507 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:45:19.980587 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:19.980592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:19.980679 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:19.980783 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:19.981600 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.982905 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.985859 (+ 2954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:19.987428 (+ 1569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:19.987619 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:19.989238 (+ 1619us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:19.989294 (+ 56us) server_negotiation.cc:299] Negotiation successful
0810 14:45:19.989452 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13521,"thread_start_us":13363,"threads_started":1}
I20260810 14:45:19.990967 2309 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f72db654cbdb4db1aa344587bc5f0589" candidate_uuid: "2f913dae965f4b248b25cf6105cc0f1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e892cc64bb7430aa632692d7ad94d5e" is_pre_election: true
I20260810 14:45:19.992313 2742 leader_election.cc:288] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [CANDIDATE]: Term 1 election: Requested vote from peers 7e892cc64bb7430aa632692d7ad94d5e (127.24.215.129:41597), 2f913dae965f4b248b25cf6105cc0f1e (127.24.215.131:41627)
I20260810 14:45:19.992918 2641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f72db654cbdb4db1aa344587bc5f0589" candidate_uuid: "76c1ea19690a4018870aea9314f752d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f913dae965f4b248b25cf6105cc0f1e" is_pre_election: true
I20260810 14:45:19.993175 2641 raft_consensus.cc:2454] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 76c1ea19690a4018870aea9314f752d8 in term 0.
I20260810 14:45:19.993274 2310 raft_consensus.cc:2454] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76c1ea19690a4018870aea9314f752d8 in term 1.
I20260810 14:45:19.993813 2641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f72db654cbdb4db1aa344587bc5f0589" candidate_uuid: "76c1ea19690a4018870aea9314f752d8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f913dae965f4b248b25cf6105cc0f1e"
I20260810 14:45:19.993743 2395 leader_election.cc:302] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76c1ea19690a4018870aea9314f752d8, 7e892cc64bb7430aa632692d7ad94d5e; no voters:
I20260810 14:45:19.994002 2641 raft_consensus.cc:3037] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:19.996140 2742 raft_consensus.cc:2781] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:45:19.996263 2742 raft_consensus.cc:686] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [term 1 LEADER]: Becoming Leader. State: Replica: 76c1ea19690a4018870aea9314f752d8, State: Running, Role: LEADER
I20260810 14:45:19.996567 2742 consensus_queue.cc:237] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:19.997486 2641 raft_consensus.cc:2454] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 76c1ea19690a4018870aea9314f752d8 in term 1.
I20260810 14:45:19.999017 2470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f72db654cbdb4db1aa344587bc5f0589" candidate_uuid: "2f913dae965f4b248b25cf6105cc0f1e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76c1ea19690a4018870aea9314f752d8" is_pre_election: true
I20260810 14:45:19.999971 2561 leader_election.cc:302] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 2f913dae965f4b248b25cf6105cc0f1e; no voters: 76c1ea19690a4018870aea9314f752d8, 7e892cc64bb7430aa632692d7ad94d5e
I20260810 14:45:20.000481 2744 raft_consensus.cc:2726] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260810 14:45:20.003556 2118 catalog_manager.cc:5495] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 76c1ea19690a4018870aea9314f752d8 (127.24.215.130). New cstate: current_term: 1 leader_uuid: "76c1ea19690a4018870aea9314f752d8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } health_report { overall_health: UNKNOWN } } }
I20260810 14:45:20.044571 2823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:20.038885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44762 (local address 127.24.215.130:38085)
0810 14:45:20.039107 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:20.039115 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:20.039136 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:45:20.039276 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:20.039279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:20.039347 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:20.039434 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:45:20.039853 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:20.040757 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:20.042820 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:20.043287 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:20.043387 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:20.043883 (+ 496us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:45:20.043995 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:45:20.044061 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:20.044228 (+ 167us) server_negotiation.cc:299] Negotiation successful
0810 14:45:20.044294 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 14:45:20.047991 2641 raft_consensus.cc:1261] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Refusing update from remote peer 76c1ea19690a4018870aea9314f752d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:45:20.048133 2310 raft_consensus.cc:1261] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e [term 1 FOLLOWER]: Refusing update from remote peer 76c1ea19690a4018870aea9314f752d8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:45:20.048650 2742 consensus_queue.cc:1035] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:20.048940 2825 consensus_queue.cc:1035] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:20.059760 2831 mvcc.cc:205] Tried to move back new op lower bound from 7316984299709566976 to 7316984299513073664. Current Snapshot: MvccSnapshot[applied={T|T < 7316984299709566976}]
I20260810 14:45:20.068039 2118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:45:20.068341 2118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:45:20.073132 2118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e: Sending DeleteTablet for 3 replicas of tablet f72db654cbdb4db1aa344587bc5f0589
I20260810 14:45:20.073958 2283 tablet_service.cc:1514] Processing DeleteTablet for tablet f72db654cbdb4db1aa344587bc5f0589 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:20 UTC) from {username='oryx', principal='oryx/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:40950
I20260810 14:45:20.074404 2617 tablet_service.cc:1514] Processing DeleteTablet for tablet f72db654cbdb4db1aa344587bc5f0589 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:20 UTC) from {username='oryx', principal='oryx/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:35790
I20260810 14:45:20.074703 2764 tablet_replica.cc:330] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e: stopping tablet replica
I20260810 14:45:20.074906 2764 raft_consensus.cc:2229] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:45:20.075176 2764 raft_consensus.cc:2258] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:20.076527 2446 tablet_service.cc:1514] Processing DeleteTablet for tablet f72db654cbdb4db1aa344587bc5f0589 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:20 UTC) from {username='oryx', principal='oryx/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:44744
I20260810 14:45:20.076975 2764 ts_tablet_manager.cc:1905] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:20.084646 2118 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34396:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:45:20.085964 2118 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:45:20.086169 2842 tablet_replica.cc:330] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e: stopping tablet replica
I20260810 14:45:20.086390 2842 raft_consensus.cc:2229] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:45:20.086654 2842 raft_consensus.cc:2258] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:20.087843 2842 ts_tablet_manager.cc:1905] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:20.095106 2842 ts_tablet_manager.cc:1918] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:45:20.095252 2842 log.cc:1196] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/f72db654cbdb4db1aa344587bc5f0589
I20260810 14:45:20.095790 2842 ts_tablet_manager.cc:1939] T f72db654cbdb4db1aa344587bc5f0589 P 7e892cc64bb7430aa632692d7ad94d5e: Deleting consensus metadata
I20260810 14:45:20.096271 2617 tablet_service.cc:1467] Processing CreateTablet for tablet 5c403afdbec445bd932f8d0714177de9 (DEFAULT_TABLE table=test-table [id=8e9a88897998456e91467c2ad981437c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:20.096310 2283 tablet_service.cc:1467] Processing CreateTablet for tablet 5c403afdbec445bd932f8d0714177de9 (DEFAULT_TABLE table=test-table [id=8e9a88897998456e91467c2ad981437c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:20.096725 2617 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c403afdbec445bd932f8d0714177de9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:20.096737 2283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c403afdbec445bd932f8d0714177de9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:20.099575 2445 tablet_service.cc:1467] Processing CreateTablet for tablet 5c403afdbec445bd932f8d0714177de9 (DEFAULT_TABLE table=test-table [id=8e9a88897998456e91467c2ad981437c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:20.100054 2445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c403afdbec445bd932f8d0714177de9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:20.100505 2764 ts_tablet_manager.cc:1918] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:45:20.100677 2764 log.cc:1196] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/f72db654cbdb4db1aa344587bc5f0589
I20260810 14:45:20.101173 2764 ts_tablet_manager.cc:1939] T f72db654cbdb4db1aa344587bc5f0589 P 2f913dae965f4b248b25cf6105cc0f1e: Deleting consensus metadata
I20260810 14:45:20.102540 2843 tablet_replica.cc:330] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8: stopping tablet replica
I20260810 14:45:20.102772 2843 raft_consensus.cc:2229] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:45:20.103196 2843 raft_consensus.cc:2258] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:20.103287 2101 catalog_manager.cc:4841] TS 2f913dae965f4b248b25cf6105cc0f1e (127.24.215.131:41627): tablet f72db654cbdb4db1aa344587bc5f0589 (table test-table [id=cae62b51af164067bc8836514435efa5]) successfully deleted
I20260810 14:45:20.103636 2103 catalog_manager.cc:4841] TS 7e892cc64bb7430aa632692d7ad94d5e (127.24.215.129:41597): tablet f72db654cbdb4db1aa344587bc5f0589 (table test-table [id=cae62b51af164067bc8836514435efa5]) successfully deleted
I20260810 14:45:20.104092 2733 tablet_bootstrap.cc:492] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e: Bootstrap starting.
I20260810 14:45:20.105140 2843 ts_tablet_manager.cc:1905] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:20.105649 2733 tablet_bootstrap.cc:654] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:20.111447 2843 ts_tablet_manager.cc:1918] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:45:20.111600 2843 log.cc:1196] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/f72db654cbdb4db1aa344587bc5f0589
I20260810 14:45:20.112150 2843 ts_tablet_manager.cc:1939] T f72db654cbdb4db1aa344587bc5f0589 P 76c1ea19690a4018870aea9314f752d8: Deleting consensus metadata
I20260810 14:45:20.113209 2104 catalog_manager.cc:4841] TS 76c1ea19690a4018870aea9314f752d8 (127.24.215.130:38085): tablet f72db654cbdb4db1aa344587bc5f0589 (table test-table [id=cae62b51af164067bc8836514435efa5]) successfully deleted
I20260810 14:45:20.115170 2737 tablet_bootstrap.cc:492] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8: Bootstrap starting.
I20260810 14:45:20.116736 2734 tablet_bootstrap.cc:492] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e: Bootstrap starting.
I20260810 14:45:20.116866 2737 tablet_bootstrap.cc:654] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:20.118600 2734 tablet_bootstrap.cc:654] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:20.127664 2737 tablet_bootstrap.cc:492] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8: No bootstrap required, opened a new log
I20260810 14:45:20.128113 2737 ts_tablet_manager.cc:1397] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.004s
I20260810 14:45:20.128315 2733 tablet_bootstrap.cc:492] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e: No bootstrap required, opened a new log
I20260810 14:45:20.128497 2733 ts_tablet_manager.cc:1397] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e: Time spent bootstrapping tablet: real 0.024s user 0.005s sys 0.018s
I20260810 14:45:20.129235 2737 raft_consensus.cc:348] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:20.129467 2737 raft_consensus.cc:374] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:20.129323 2733 raft_consensus.cc:348] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:20.129552 2737 raft_consensus.cc:729] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c1ea19690a4018870aea9314f752d8, State: Initialized, Role: FOLLOWER
I20260810 14:45:20.129585 2733 raft_consensus.cc:374] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:20.129662 2733 raft_consensus.cc:729] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f913dae965f4b248b25cf6105cc0f1e, State: Initialized, Role: FOLLOWER
I20260810 14:45:20.129837 2737 consensus_queue.cc:260] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:20.129899 2733 consensus_queue.cc:260] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:20.130425 2733 ts_tablet_manager.cc:1428] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260810 14:45:20.131903 2737 ts_tablet_manager.cc:1428] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8: Time spent starting tablet: real 0.004s user 0.002s sys 0.000s
I20260810 14:45:20.133324 2734 tablet_bootstrap.cc:492] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e: No bootstrap required, opened a new log
I20260810 14:45:20.133513 2734 ts_tablet_manager.cc:1397] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.000s
I20260810 14:45:20.134359 2734 raft_consensus.cc:348] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:20.134639 2734 raft_consensus.cc:374] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:20.134714 2734 raft_consensus.cc:729] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e892cc64bb7430aa632692d7ad94d5e, State: Initialized, Role: FOLLOWER
I20260810 14:45:20.134964 2734 consensus_queue.cc:260] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:20.135543 2734 ts_tablet_manager.cc:1428] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260810 14:45:20.159524 2774 raft_consensus.cc:482] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:45:20.159759 2774 raft_consensus.cc:504] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:20.160490 2774 leader_election.cc:288] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 76c1ea19690a4018870aea9314f752d8 (127.24.215.130:38085), 2f913dae965f4b248b25cf6105cc0f1e (127.24.215.131:41627)
I20260810 14:45:20.161010 2470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c403afdbec445bd932f8d0714177de9" candidate_uuid: "7e892cc64bb7430aa632692d7ad94d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76c1ea19690a4018870aea9314f752d8" is_pre_election: true
I20260810 14:45:20.161309 2470 raft_consensus.cc:2454] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e892cc64bb7430aa632692d7ad94d5e in term 0.
I20260810 14:45:20.161239 2641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c403afdbec445bd932f8d0714177de9" candidate_uuid: "7e892cc64bb7430aa632692d7ad94d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f913dae965f4b248b25cf6105cc0f1e" is_pre_election: true
I20260810 14:45:20.161506 2641 raft_consensus.cc:2454] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7e892cc64bb7430aa632692d7ad94d5e in term 0.
I20260810 14:45:20.161832 2226 leader_election.cc:302] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76c1ea19690a4018870aea9314f752d8, 7e892cc64bb7430aa632692d7ad94d5e; no voters:
I20260810 14:45:20.162164 2774 raft_consensus.cc:2781] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:45:20.162282 2774 raft_consensus.cc:482] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:45:20.162411 2774 raft_consensus.cc:3037] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:20.164608 2774 raft_consensus.cc:504] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:20.165227 2774 leader_election.cc:288] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [CANDIDATE]: Term 1 election: Requested vote from peers 76c1ea19690a4018870aea9314f752d8 (127.24.215.130:38085), 2f913dae965f4b248b25cf6105cc0f1e (127.24.215.131:41627)
I20260810 14:45:20.165705 2470 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c403afdbec445bd932f8d0714177de9" candidate_uuid: "7e892cc64bb7430aa632692d7ad94d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "76c1ea19690a4018870aea9314f752d8"
I20260810 14:45:20.165802 2641 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c403afdbec445bd932f8d0714177de9" candidate_uuid: "7e892cc64bb7430aa632692d7ad94d5e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f913dae965f4b248b25cf6105cc0f1e"
I20260810 14:45:20.165985 2470 raft_consensus.cc:3037] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:20.166018 2641 raft_consensus.cc:3037] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:20.169065 2641 raft_consensus.cc:2454] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e892cc64bb7430aa632692d7ad94d5e in term 1.
I20260810 14:45:20.169054 2470 raft_consensus.cc:2454] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7e892cc64bb7430aa632692d7ad94d5e in term 1.
I20260810 14:45:20.169488 2226 leader_election.cc:302] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 76c1ea19690a4018870aea9314f752d8, 7e892cc64bb7430aa632692d7ad94d5e; no voters:
I20260810 14:45:20.169785 2774 raft_consensus.cc:2781] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:45:20.170287 2774 raft_consensus.cc:686] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [term 1 LEADER]: Becoming Leader. State: Replica: 7e892cc64bb7430aa632692d7ad94d5e, State: Running, Role: LEADER
I20260810 14:45:20.170585 2774 consensus_queue.cc:237] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } }
I20260810 14:45:20.173331 2119 catalog_manager.cc:5495] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e reported cstate change: term changed from 0 to 1, leader changed from <none> to 7e892cc64bb7430aa632692d7ad94d5e (127.24.215.129). New cstate: current_term: 1 leader_uuid: "7e892cc64bb7430aa632692d7ad94d5e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e892cc64bb7430aa632692d7ad94d5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41597 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 } health_report { overall_health: UNKNOWN } } }
I20260810 14:45:20.217345 2824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:20.209546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40972 (local address 127.24.215.129:41597)
0810 14:45:20.209764 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:20.209773 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:20.209803 (+ 30us) server_negotiation.cc:407] Connection header received
0810 14:45:20.209927 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:20.209936 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:20.210051 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:20.210181 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:45:20.210644 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:20.211913 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:20.214659 (+ 2746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:20.215358 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:20.215498 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:20.216108 (+ 610us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:45:20.216278 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:45:20.216374 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:20.216938 (+ 564us) server_negotiation.cc:299] Negotiation successful
0810 14:45:20.217017 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 14:45:20.235889 2641 raft_consensus.cc:1261] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Refusing update from remote peer 7e892cc64bb7430aa632692d7ad94d5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:45:20.235931 2470 raft_consensus.cc:1261] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8 [term 1 FOLLOWER]: Refusing update from remote peer 7e892cc64bb7430aa632692d7ad94d5e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:45:20.236572 2850 consensus_queue.cc:1035] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f913dae965f4b248b25cf6105cc0f1e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:20.236995 2851 consensus_queue.cc:1035] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [LEADER]: Connected to new peer: Peer: permanent_uuid: "76c1ea19690a4018870aea9314f752d8" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 38085 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:20.294346 2118 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:45:20.294620 2118 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34396:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:45:20.297605 2118 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e34e7455f58e4638ae3fad80b7500e2e: Sending DeleteTablet for 3 replicas of tablet 5c403afdbec445bd932f8d0714177de9
I20260810 14:45:20.298372 2283 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c403afdbec445bd932f8d0714177de9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:20 UTC) from {username='oryx', principal='oryx/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:40950
I20260810 14:45:20.298529 2445 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c403afdbec445bd932f8d0714177de9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:20 UTC) from {username='oryx', principal='oryx/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:44744
I20260810 14:45:20.298867 2842 tablet_replica.cc:330] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e: stopping tablet replica
I20260810 14:45:20.298960 2843 tablet_replica.cc:330] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8: stopping tablet replica
I20260810 14:45:20.299116 2842 raft_consensus.cc:2229] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:45:20.299206 2843 raft_consensus.cc:2229] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:45:20.299422 2843 raft_consensus.cc:2258] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:20.299479 2842 raft_consensus.cc:2258] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:20.299746 2617 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c403afdbec445bd932f8d0714177de9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:20 UTC) from {username='oryx', principal='oryx/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:35790
I20260810 14:45:20.300053 2764 tablet_replica.cc:330] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e: stopping tablet replica
I20260810 14:45:20.300302 2764 raft_consensus.cc:2229] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:45:20.300531 2764 raft_consensus.cc:2258] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:20.300935 2842 ts_tablet_manager.cc:1905] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:20.301414 2843 ts_tablet_manager.cc:1905] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:20.301661 2764 ts_tablet_manager.cc:1905] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:20.302776 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 2205
I20260810 14:45:20.308269 2764 ts_tablet_manager.cc:1918] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:45:20.308255 2843 ts_tablet_manager.cc:1918] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:45:20.308444 2764 log.cc:1196] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/5c403afdbec445bd932f8d0714177de9
I20260810 14:45:20.308444 2843 log.cc:1196] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/5c403afdbec445bd932f8d0714177de9
I20260810 14:45:20.308871 2764 ts_tablet_manager.cc:1939] T 5c403afdbec445bd932f8d0714177de9 P 2f913dae965f4b248b25cf6105cc0f1e: Deleting consensus metadata
I20260810 14:45:20.308868 2843 ts_tablet_manager.cc:1939] T 5c403afdbec445bd932f8d0714177de9 P 76c1ea19690a4018870aea9314f752d8: Deleting consensus metadata
I20260810 14:45:20.309949 2104 catalog_manager.cc:4841] TS 76c1ea19690a4018870aea9314f752d8 (127.24.215.130:38085): tablet 5c403afdbec445bd932f8d0714177de9 (table test-table [id=8e9a88897998456e91467c2ad981437c]) successfully deleted
I20260810 14:45:20.310397 2101 catalog_manager.cc:4841] TS 2f913dae965f4b248b25cf6105cc0f1e (127.24.215.131:41627): tablet 5c403afdbec445bd932f8d0714177de9 (table test-table [id=8e9a88897998456e91467c2ad981437c]) successfully deleted
I20260810 14:45:20.313011 2842 ts_tablet_manager.cc:1918] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260810 14:45:20.313184 2842 log.cc:1196] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/5c403afdbec445bd932f8d0714177de9
I20260810 14:45:20.313649 2842 ts_tablet_manager.cc:1939] T 5c403afdbec445bd932f8d0714177de9 P 7e892cc64bb7430aa632692d7ad94d5e: Deleting consensus metadata
I20260810 14:45:20.314599 2103 catalog_manager.cc:4841] TS 7e892cc64bb7430aa632692d7ad94d5e (127.24.215.129:41597): tablet 5c403afdbec445bd932f8d0714177de9 (table test-table [id=8e9a88897998456e91467c2ad981437c]) successfully deleted
Aug 10 14:45:20 dist-test-slave-vkjp krb5kdc[2067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.215.129@KRBTEST.COM
I20260810 14:45:20.327244 2824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:20.309639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40986 (local address 127.24.215.129:41597)
0810 14:45:20.309813 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:20.309819 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:20.310331 (+ 512us) server_negotiation.cc:407] Connection header received
0810 14:45:20.310569 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:20.310582 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:20.310646 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:20.310723 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:20.312489 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:20.313771 (+ 1282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:20.315637 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:20.316031 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:20.316128 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:20.320953 (+ 4825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:20.320969 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:20.320986 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:20.321042 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:20.324277 (+ 3235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:20.325030 (+ 753us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:20.325038 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:20.325041 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:20.325183 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:20.325592 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:20.325596 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:20.325598 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:20.326006 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:20.326347 (+ 341us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:20.326601 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 14:45:20.326824 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 14:45:20.327931 2360 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:20.487774 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 2205
I20260810 14:45:20.520154 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 2370
Aug 10 14:45:20 dist-test-slave-vkjp krb5kdc[2067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.215.130@KRBTEST.COM
I20260810 14:45:20.535590 2823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:20.520998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35922 (local address 127.24.215.130:38085)
0810 14:45:20.521208 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:20.521213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:20.521233 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:45:20.521305 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:20.521311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:20.521392 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:20.521492 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:20.522781 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:20.523784 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:20.525070 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:20.525475 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:20.525572 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:20.529820 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:20.529835 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:20.529853 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:20.529896 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:20.532953 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:20.533694 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:20.533701 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:20.533703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:20.533843 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:20.534200 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:20.534204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:20.534206 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:20.534589 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:20.534812 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:20.535067 (+ 255us) server_negotiation.cc:299] Negotiation successful
0810 14:45:20.535316 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 14:45:20.536290 2529 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:20.661763 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 2370
I20260810 14:45:20.689919 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 2538
Aug 10 14:45:20 dist-test-slave-vkjp krb5kdc[2067](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373119, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.24.215.131@KRBTEST.COM
I20260810 14:45:20.705380 2861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:20.690762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48126 (local address 127.24.215.131:41627)
0810 14:45:20.691340 (+ 578us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:20.691345 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:20.691362 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:45:20.691420 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:20.691424 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:20.691483 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:20.691576 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:20.692821 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:20.693542 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:20.694774 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:20.695278 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:20.695381 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:20.699743 (+ 4362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:20.699757 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:20.699774 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:20.699811 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:20.702734 (+ 2923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:20.703507 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:20.703514 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:20.703517 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:20.703649 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:20.703976 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:20.703980 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:20.703982 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:20.704360 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:20.704575 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:20.704842 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 14:45:20.705020 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":479,"thread_start_us":341,"threads_started":1}
I20260810 14:45:20.706037 2699 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:20.804483 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 2538
I20260810 14:45:20.825721 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 2083
I20260810 14:45:20.826889 2163 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:20.947211 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 2083
2026-08-10T14:45:20Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (5834 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:45:21 dist-test-slave-vkjp krb5kdc[2866](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:45:21 dist-test-slave-vkjp krb5kdc[2866](info): set up 2 sockets
Aug 10 14:45:21 dist-test-slave-vkjp krb5kdc[2866](info): commencing operation
krb5kdc: starting...
W20260810 14:45:22.136092 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:45:22 dist-test-slave-vkjp krb5kdc[2866](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:45:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:45:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:45:22.386943 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:41955
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:46451
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:41955
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:22.721050 2882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:22.721360 2882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:22.721519 2882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:22.728605 2882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:45:22.728718 2882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:22.728840 2882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:22.728928 2882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:45:22.729012 2882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:45:22.738722 2882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:46451
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:41955
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:41955
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:22.740059 2882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:22.741369 2882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:22.753247 2890 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:22.753218 2894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:22.753185 2891 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:22.754479 2882 server_base.cc:1034] running on GCE node
I20260810 14:45:22.755280 2882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:22.756590 2882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:22.757827 2882 hybrid_clock.cc:648] HybridClock initialized: now 1786373122757779 us; error 53 us; skew 500 ppm
I20260810 14:45:22.758177 2882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-10T14:45:22Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1885 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:45:22 dist-test-slave-vkjp krb5kdc[2902](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:45:22 dist-test-slave-vkjp krb5kdc[2902](info): set up 2 sockets
Aug 10 14:45:22 dist-test-slave-vkjp krb5kdc[2902](info): commencing operation
krb5kdc: starting...
W20260810 14:45:24.033938 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.000s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:45:24 dist-test-slave-vkjp krb5kdc[2902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:45:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:45:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:45:24.282773 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:34891
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:34891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:24.617173 2917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:24.617440 2917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:24.617533 2917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:24.624747 2917 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:45:24.624857 2917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:24.624912 2917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:24.624976 2917 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:45:24.625021 2917 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:45:24.634851 2917 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:34891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:34891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:24.636257 2917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:24.637562 2917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:24.649536 2925 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:24.649555 2923 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:24.649590 2922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:24.650573 2917 server_base.cc:1034] running on GCE node
I20260810 14:45:24.651396 2917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:24.652740 2917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:24.653976 2917 hybrid_clock.cc:648] HybridClock initialized: now 1786373124653926 us; error 35 us; skew 500 ppm
I20260810 14:45:24.654323 2917 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:24 dist-test-slave-vkjp krb5kdc[2902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373124, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:24.657375 2917 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:45:24.658860 2917 webserver.cc:460] Webserver started at http://127.24.215.190:38385/ using document root <none> and password file <none>
I20260810 14:45:24.659580 2917 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:24.659683 2917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:24.659957 2917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:24.662883 2917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "707d39b966c04eae8ce44409e69fed0b"
format_stamp: "Formatted at 2026-08-10 14:45:24 on dist-test-slave-vkjp"
server_key: "d3c20c9436ee91f5acf3599e6ecee843"
server_key_iv: "d8a1c0098f46958a55be0f781b3797d7"
server_key_version: "encryptionkey@0"
I20260810 14:45:24.663698 2917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "707d39b966c04eae8ce44409e69fed0b"
format_stamp: "Formatted at 2026-08-10 14:45:24 on dist-test-slave-vkjp"
server_key: "d3c20c9436ee91f5acf3599e6ecee843"
server_key_iv: "d8a1c0098f46958a55be0f781b3797d7"
server_key_version: "encryptionkey@0"
I20260810 14:45:24.668665 2917 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:45:24.671911 2935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:24.673169 2917 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:45:24.673348 2917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "707d39b966c04eae8ce44409e69fed0b"
format_stamp: "Formatted at 2026-08-10 14:45:24 on dist-test-slave-vkjp"
server_key: "d3c20c9436ee91f5acf3599e6ecee843"
server_key_iv: "d8a1c0098f46958a55be0f781b3797d7"
server_key_version: "encryptionkey@0"
I20260810 14:45:24.673550 2917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:24.685695 2917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:24.689554 2917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:24.689850 2917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:24.708905 2917 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:34891
I20260810 14:45:24.708942 2997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:34891 every 8 connection(s)
I20260810 14:45:24.710366 2917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:45:24.714167 2998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:24.716495 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 2917
I20260810 14:45:24.716734 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:45:24.717563 25438 external_mini_cluster.cc:1428] Setting key f9e826be1cc4bbdf86d973b444e4c269
I20260810 14:45:24.729432 2998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b: Bootstrap starting.
Aug 10 14:45:24 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:45:24.732939 2998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:24.734179 2998 log.cc:824] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:24.737389 2998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b: No bootstrap required, opened a new log
I20260810 14:45:24.740651 3001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:24.719815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46090 (local address 127.24.215.190:34891)
0810 14:45:24.720620 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:24.720632 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:24.720673 (+ 41us) server_negotiation.cc:407] Connection header received
0810 14:45:24.722320 (+ 1647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:24.722343 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:24.722728 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:24.723147 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:24.724642 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:24.726044 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:24.727686 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:24.728767 (+ 1081us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:24.729003 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:24.733320 (+ 4317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:24.733328 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:24.733342 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:24.733379 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:24.736508 (+ 3129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:24.737353 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:24.737358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:24.737364 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:24.737577 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:24.737999 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:24.738002 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:24.738004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:24.738471 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:24.738709 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:24.739282 (+ 573us) server_negotiation.cc:299] Negotiation successful
0810 14:45:24.739671 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":347,"threads_started":1}
I20260810 14:45:24.745944 2998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707d39b966c04eae8ce44409e69fed0b" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34891 } }
I20260810 14:45:24.746307 2998 raft_consensus.cc:374] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:24.746457 2998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707d39b966c04eae8ce44409e69fed0b, State: Initialized, Role: FOLLOWER
I20260810 14:45:24.747022 2998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707d39b966c04eae8ce44409e69fed0b" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34891 } }
I20260810 14:45:24.747363 2998 raft_consensus.cc:388] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:45:24.747521 2998 raft_consensus.cc:482] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:45:24.747694 2998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:24.750128 2998 raft_consensus.cc:504] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707d39b966c04eae8ce44409e69fed0b" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34891 } }
I20260810 14:45:24.750564 2998 leader_election.cc:302] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 707d39b966c04eae8ce44409e69fed0b; no voters:
I20260810 14:45:24.751073 2998 leader_election.cc:288] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:45:24.751271 3003 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:45:24.751946 3003 raft_consensus.cc:686] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 1 LEADER]: Becoming Leader. State: Replica: 707d39b966c04eae8ce44409e69fed0b, State: Running, Role: LEADER
I20260810 14:45:24.752460 3003 consensus_queue.cc:237] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707d39b966c04eae8ce44409e69fed0b" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34891 } }
I20260810 14:45:24.753289 2998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:45:24.755339 3006 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 707d39b966c04eae8ce44409e69fed0b. Latest consensus state: current_term: 1 leader_uuid: "707d39b966c04eae8ce44409e69fed0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707d39b966c04eae8ce44409e69fed0b" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34891 } } }
I20260810 14:45:24.755818 3006 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:24.756259 3004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "707d39b966c04eae8ce44409e69fed0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707d39b966c04eae8ce44409e69fed0b" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34891 } } }
I20260810 14:45:24.756599 3004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:24.756736 3012 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:45:24.762174 3012 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:45:24.771286 3012 catalog_manager.cc:1347] Generated new cluster ID: eb482597a4494205a73ca99cc5c03f58
I20260810 14:45:24.771405 3012 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:45:24.796737 3012 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:45:24.798775 3012 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:45:24.810384 3012 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b: Generated new TSK 0
I20260810 14:45:24.811496 3012 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:45:24.886700 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34891
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:25.224637 3029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:25.224921 3029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:25.225016 3029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:25.232348 3029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:25.232468 3029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:25.232582 3029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:45:25.242465 3029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:25.243768 3029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:25.245154 3029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:25.258948 3035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:25.258956 3037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:25.259207 3029 server_base.cc:1034] running on GCE node
W20260810 14:45:25.259503 3039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:25.260210 3029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:25.261193 3029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:25.262396 3029 hybrid_clock.cc:648] HybridClock initialized: now 1786373125262348 us; error 44 us; skew 500 ppm
I20260810 14:45:25.262715 3029 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:25 dist-test-slave-vkjp krb5kdc[2902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373125, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:25.266063 3029 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:45:25.267688 3029 webserver.cc:460] Webserver started at http://127.24.215.129:44295/ using document root <none> and password file <none>
I20260810 14:45:25.268401 3029 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:25.268533 3029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:25.268859 3029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:25.271978 3029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "18699830eb914492ad1a804f9a21bb13"
format_stamp: "Formatted at 2026-08-10 14:45:25 on dist-test-slave-vkjp"
server_key: "c5dde22fa130b7689caa815197302df0"
server_key_iv: "ec74e367233f8ebc79694c4671c050b0"
server_key_version: "encryptionkey@0"
I20260810 14:45:25.272794 3029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "18699830eb914492ad1a804f9a21bb13"
format_stamp: "Formatted at 2026-08-10 14:45:25 on dist-test-slave-vkjp"
server_key: "c5dde22fa130b7689caa815197302df0"
server_key_iv: "ec74e367233f8ebc79694c4671c050b0"
server_key_version: "encryptionkey@0"
I20260810 14:45:25.278383 3029 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260810 14:45:25.282135 3049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:25.283560 3029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:45:25.283743 3029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "18699830eb914492ad1a804f9a21bb13"
format_stamp: "Formatted at 2026-08-10 14:45:25 on dist-test-slave-vkjp"
server_key: "c5dde22fa130b7689caa815197302df0"
server_key_iv: "ec74e367233f8ebc79694c4671c050b0"
server_key_version: "encryptionkey@0"
I20260810 14:45:25.283957 3029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:25.317551 3029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:25.322160 3029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:25.322520 3029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:25.323599 3029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:25.325037 3029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:25.325138 3029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:25.325284 3029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:25.325354 3029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:25.360257 3029 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:43687
I20260810 14:45:25.360316 3177 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:43687 every 8 connection(s)
I20260810 14:45:25.361599 3029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:45:25.363534 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 3029
I20260810 14:45:25.363752 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:45:25.364542 25438 external_mini_cluster.cc:1428] Setting key eff7c8058b1a9d42b680ab7bbd1a07da
Aug 10 14:45:25 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373125, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:45:25.382392 3182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:25.364292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:44163 (local address 127.24.215.190:34891)
0810 14:45:25.364888 (+ 596us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:25.364895 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:25.365824 (+ 929us) server_negotiation.cc:407] Connection header received
0810 14:45:25.367799 (+ 1975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:25.367804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:25.367891 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:25.368014 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:25.369509 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:25.370249 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:25.371877 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:25.372268 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:25.372405 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:25.376156 (+ 3751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:25.376163 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:25.376175 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:25.376214 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:25.378939 (+ 2725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:25.379955 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:25.379959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:25.379961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:25.380076 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:25.380569 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:25.380573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:25.380575 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:25.380902 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:25.381017 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:25.381965 (+ 948us) server_negotiation.cc:299] Negotiation successful
0810 14:45:25.382138 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":251,"threads_started":1}
I20260810 14:45:25.383991 3183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:25.364845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:44163)
0810 14:45:25.365653 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:25.365703 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:25.367540 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:25.368252 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:25.368260 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:25.368764 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:25.369327 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:25.369344 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:25.370417 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:25.370421 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:25.371715 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:25.371728 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:25.372513 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:25.372522 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:25.372755 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:25.373456 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:25.373482 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:25.375883 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:25.379112 (+ 3229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:25.379125 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:25.379144 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:25.379827 (+ 683us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:25.380183 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:25.380191 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:25.380197 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:25.380438 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:25.381060 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:25.381071 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:25.381543 (+ 472us) client_negotiation.cc:769] Sending connection context
0810 14:45:25.381849 (+ 306us) client_negotiation.cc:240] Negotiation successful
0810 14:45:25.382424 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":347,"threads_started":1}
I20260810 14:45:25.385587 3180 heartbeater.cc:347] Connected to a master server at 127.24.215.190:34891
I20260810 14:45:25.385924 3180 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:25.386854 3180 heartbeater.cc:511] Master 127.24.215.190:34891 requested a full tablet report, sending...
I20260810 14:45:25.389206 2953 ts_manager.cc:194] Registered new tserver with Master: 18699830eb914492ad1a804f9a21bb13 (127.24.215.129:43687)
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
I20260810 14:45:25.390718 2953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:44163
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:45:25.422906 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34891
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:25.758280 3188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:25.758595 3188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:25.758769 3188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:25.766047 3188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:25.766189 3188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:25.766360 3188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:45:25.775939 3188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:25.777387 3188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:25.778834 3188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:25.792171 3193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:25.792196 3194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:25.792564 3196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:25.797179 3188 server_base.cc:1034] running on GCE node
I20260810 14:45:25.797894 3188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:25.798836 3188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:25.800060 3188 hybrid_clock.cc:648] HybridClock initialized: now 1786373125799991 us; error 63 us; skew 500 ppm
I20260810 14:45:25.800390 3188 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:25 dist-test-slave-vkjp krb5kdc[2902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373125, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:25.803557 3188 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:45:25.805004 3188 webserver.cc:460] Webserver started at http://127.24.215.130:37905/ using document root <none> and password file <none>
I20260810 14:45:25.805719 3188 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:25.805864 3188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:25.806182 3188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:25.809113 3188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "0e69c71ec977486897afec01b1878ea5"
format_stamp: "Formatted at 2026-08-10 14:45:25 on dist-test-slave-vkjp"
server_key: "74c79f524aca207036dfc7dad4669219"
server_key_iv: "dcd774a3761833fc5c1bddef36713175"
server_key_version: "encryptionkey@0"
I20260810 14:45:25.809907 3188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "0e69c71ec977486897afec01b1878ea5"
format_stamp: "Formatted at 2026-08-10 14:45:25 on dist-test-slave-vkjp"
server_key: "74c79f524aca207036dfc7dad4669219"
server_key_iv: "dcd774a3761833fc5c1bddef36713175"
server_key_version: "encryptionkey@0"
I20260810 14:45:25.814926 3188 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:45:25.818297 3206 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:25.819636 3188 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:45:25.819815 3188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "0e69c71ec977486897afec01b1878ea5"
format_stamp: "Formatted at 2026-08-10 14:45:25 on dist-test-slave-vkjp"
server_key: "74c79f524aca207036dfc7dad4669219"
server_key_iv: "dcd774a3761833fc5c1bddef36713175"
server_key_version: "encryptionkey@0"
I20260810 14:45:25.820025 3188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:25.843364 3188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:25.847913 3188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:25.848217 3188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:25.849206 3188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:25.850553 3188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:25.850660 3188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:25.850813 3188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:25.850893 3188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:25.884328 3188 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:43047
I20260810 14:45:25.884413 3343 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:43047 every 8 connection(s)
I20260810 14:45:25.885700 3188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:45:25.887583 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 3188
I20260810 14:45:25.887804 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:45:25.888320 25438 external_mini_cluster.cc:1428] Setting key 5eedb57860e00a5a1cf5edf0fe4cb833
Aug 10 14:45:25 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373125, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:45:25.906394 3346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:25.888503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:51415 (local address 127.24.215.190:34891)
0810 14:45:25.888996 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:25.889001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:25.891154 (+ 2153us) server_negotiation.cc:407] Connection header received
0810 14:45:25.892127 (+ 973us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:25.892132 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:25.892220 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:25.892332 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:25.893814 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:25.894571 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:25.896162 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:25.896571 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:25.896696 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:25.900443 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:25.900448 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:25.900452 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:25.900486 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:25.903047 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:25.904084 (+ 1037us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:25.904088 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:25.904090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:25.904221 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:25.904701 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:25.904704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:25.904706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:25.904955 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:25.905073 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:25.905973 (+ 900us) server_negotiation.cc:299] Negotiation successful
0810 14:45:25.906141 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":227,"threads_started":1}
I20260810 14:45:25.907997 3347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:25.889157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:51415)
0810 14:45:25.890014 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:25.890065 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:25.891873 (+ 1808us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:25.892573 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:25.892581 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:25.893075 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:25.893645 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:25.893661 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:25.894729 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:25.894741 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:25.896010 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:25.896023 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:25.896799 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:25.896806 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:25.897018 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:25.897779 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:25.897801 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:25.900204 (+ 2403us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:25.903250 (+ 3046us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:25.903263 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:25.903285 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:25.903947 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:25.904328 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:25.904335 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:25.904342 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:25.904590 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:25.905109 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:25.905119 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:25.905570 (+ 451us) client_negotiation.cc:769] Sending connection context
0810 14:45:25.905874 (+ 304us) client_negotiation.cc:240] Negotiation successful
0810 14:45:25.906484 (+ 610us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":581,"thread_start_us":338,"threads_started":1}
I20260810 14:45:25.909521 3344 heartbeater.cc:347] Connected to a master server at 127.24.215.190:34891
I20260810 14:45:25.909842 3344 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:25.910573 3344 heartbeater.cc:511] Master 127.24.215.190:34891 requested a full tablet report, sending...
I20260810 14:45:25.912103 2953 ts_manager.cc:194] Registered new tserver with Master: 0e69c71ec977486897afec01b1878ea5 (127.24.215.130:43047)
I20260810 14:45:25.912917 2953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:51415
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:45:25.946795 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34891
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:26.280128 3350 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:26.280400 3350 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:26.280493 3350 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:26.287559 3350 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:26.287664 3350 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:26.287842 3350 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:45:26.297518 3350 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:26.298808 3350 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:26.300143 3350 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:26.313760 3355 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:26.313798 3356 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:26.313839 3358 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:26.319396 3350 server_base.cc:1034] running on GCE node
I20260810 14:45:26.320051 3350 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:26.321015 3350 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:26.322260 3350 hybrid_clock.cc:648] HybridClock initialized: now 1786373126322216 us; error 54 us; skew 500 ppm
I20260810 14:45:26.322582 3350 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:26 dist-test-slave-vkjp krb5kdc[2902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373126, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:26.325884 3350 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:45:26.327457 3350 webserver.cc:460] Webserver started at http://127.24.215.131:34811/ using document root <none> and password file <none>
I20260810 14:45:26.328125 3350 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:26.328223 3350 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:26.328478 3350 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:26.331449 3350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "5f0efa1d27f344c7addd3f06affc103d"
format_stamp: "Formatted at 2026-08-10 14:45:26 on dist-test-slave-vkjp"
server_key: "f5169af1d45bb6a6dd2e16fc18da12ba"
server_key_iv: "c35cedb93f6b0d002971cfa73c5bca50"
server_key_version: "encryptionkey@0"
I20260810 14:45:26.332185 3350 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "5f0efa1d27f344c7addd3f06affc103d"
format_stamp: "Formatted at 2026-08-10 14:45:26 on dist-test-slave-vkjp"
server_key: "f5169af1d45bb6a6dd2e16fc18da12ba"
server_key_iv: "c35cedb93f6b0d002971cfa73c5bca50"
server_key_version: "encryptionkey@0"
I20260810 14:45:26.337106 3350 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:45:26.340421 3365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:26.341686 3350 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:45:26.341807 3350 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "5f0efa1d27f344c7addd3f06affc103d"
format_stamp: "Formatted at 2026-08-10 14:45:26 on dist-test-slave-vkjp"
server_key: "f5169af1d45bb6a6dd2e16fc18da12ba"
server_key_iv: "c35cedb93f6b0d002971cfa73c5bca50"
server_key_version: "encryptionkey@0"
I20260810 14:45:26.341986 3350 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:26.358793 3350 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:26.363147 3350 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:26.363433 3350 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:26.364351 3350 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:26.365620 3350 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:26.365721 3350 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:26.365869 3350 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:26.365947 3350 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:26.393731 3180 heartbeater.cc:503] Master 127.24.215.190:34891 was elected leader, sending a full tablet report...
I20260810 14:45:26.403306 3350 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:44287
I20260810 14:45:26.403339 3505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:44287 every 8 connection(s)
I20260810 14:45:26.404584 3350 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:45:26.410995 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 3350
I20260810 14:45:26.411216 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:45:26.411675 25438 external_mini_cluster.cc:1428] Setting key df3cb0dbfe719c8cf7043cd632f03890
Aug 10 14:45:26 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373126, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:45:26.424352 3508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:26.407113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:44093 (local address 127.24.215.190:34891)
0810 14:45:26.407556 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:26.407560 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:26.408487 (+ 927us) server_negotiation.cc:407] Connection header received
0810 14:45:26.410293 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:26.410298 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:26.410389 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:26.410514 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:26.412248 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:26.413178 (+ 930us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:26.415268 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:26.415662 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:26.415762 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:26.418975 (+ 3213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:26.418980 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:26.418984 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:26.419027 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:26.421588 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:26.422276 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:26.422280 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:26.422283 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:26.422398 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:26.422775 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:26.422779 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:26.422781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:26.423052 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:26.423234 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:26.423908 (+ 674us) server_negotiation.cc:299] Negotiation successful
0810 14:45:26.424093 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":200,"threads_started":1}
I20260810 14:45:26.425406 3509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:26.407640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:44093)
0810 14:45:26.408332 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:26.408368 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:26.410027 (+ 1659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:26.410766 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:26.410773 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:26.411448 (+ 675us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:26.412057 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:26.412075 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:26.413335 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:26.413339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:26.415142 (+ 1803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:26.415156 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:26.415848 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:26.415853 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:26.416060 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:26.416655 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:26.416680 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:26.418761 (+ 2081us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:26.421710 (+ 2949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:26.421717 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:26.421728 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:26.422161 (+ 433us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:26.422491 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:26.422494 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:26.422496 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:26.422664 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:26.423221 (+ 557us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:26.423227 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:26.423570 (+ 343us) client_negotiation.cc:769] Sending connection context
0810 14:45:26.423807 (+ 237us) client_negotiation.cc:240] Negotiation successful
0810 14:45:26.424241 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":274,"threads_started":1}
I20260810 14:45:26.426795 3506 heartbeater.cc:347] Connected to a master server at 127.24.215.190:34891
I20260810 14:45:26.427156 3506 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:26.427883 3506 heartbeater.cc:511] Master 127.24.215.190:34891 requested a full tablet report, sending...
I20260810 14:45:26.429267 2953 ts_manager.cc:194] Registered new tserver with Master: 5f0efa1d27f344c7addd3f06affc103d (127.24.215.131:44287)
I20260810 14:45:26.430076 2953 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:44093
I20260810 14:45:26.439191 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.129.keytab.
Entry for principal oryx/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.129.keytab.
WARNING: no policy specified for oryx/127.24.215.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.215.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.130.keytab.
Entry for principal oryx/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.130.keytab.
WARNING: no policy specified for oryx/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.131.keytab.
Entry for principal oryx/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.131.keytab.
I20260810 14:45:26.593745 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 3029
Aug 10 14:45:26 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:45:26.610348 3520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:26.594920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55314 (local address 127.24.215.129:43687)
0810 14:45:26.595812 (+ 892us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:26.595820 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:26.595848 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:45:26.595927 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:26.595934 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:26.596171 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:26.596342 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:26.597707 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:26.598754 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:26.600079 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:26.600698 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:26.600811 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:26.604901 (+ 4090us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:26.604913 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:26.604939 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:26.604974 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:26.607741 (+ 2767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:26.608461 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:26.608498 (+ 37us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:26.608501 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:26.608664 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:26.609014 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:26.609018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:26.609020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:26.609385 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:26.609618 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:26.609860 (+ 242us) server_negotiation.cc:299] Negotiation successful
0810 14:45:26.610064 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":715,"thread_start_us":548,"threads_started":1}
I20260810 14:45:26.611678 3173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:26.694360 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 3029
I20260810 14:45:26.717540 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 3188
Aug 10 14:45:26 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:45:26.733551 3523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:26.718494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45652 (local address 127.24.215.130:43047)
0810 14:45:26.719024 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:26.719031 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:26.719055 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:45:26.719164 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:26.719174 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:26.719482 (+ 308us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:26.719658 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:26.720911 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:26.721923 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:26.723248 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:26.723783 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:26.723932 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:26.728221 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:26.728232 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:26.728246 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:26.728286 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:26.731280 (+ 2994us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:26.731854 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:26.731860 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:26.731862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:26.731984 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:26.732284 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:26.732287 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:26.732289 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:26.732628 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:26.732860 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:26.733065 (+ 205us) server_negotiation.cc:299] Negotiation successful
0810 14:45:26.733254 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":287,"threads_started":1}
I20260810 14:45:26.734776 3339 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:26.816911 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 3188
I20260810 14:45:26.837016 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 3350
Aug 10 14:45:26 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:45:26.853459 3525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:26.837935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59616 (local address 127.24.215.131:44287)
0810 14:45:26.838489 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:26.838496 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:26.838525 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:45:26.838583 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:26.838590 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:26.838796 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:26.838973 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:26.840359 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:26.841348 (+ 989us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:26.842620 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:26.843245 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:26.843382 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:26.848190 (+ 4808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:26.848201 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:26.848217 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:26.848250 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:26.850830 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:26.851619 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:26.851626 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:26.851628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:26.851750 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:26.852083 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:26.852086 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:26.852088 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:26.852426 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:26.852660 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:26.852880 (+ 220us) server_negotiation.cc:299] Negotiation successful
0810 14:45:26.853084 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":247,"threads_started":1}
I20260810 14:45:26.854816 3501 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:26.937971 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 3350
I20260810 14:45:26.958151 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 2917
I20260810 14:45:26.959497 2990 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:27.064680 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 2917
I20260810 14:45:27.088562 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:34891
--webserver_interface=127.24.215.190
--webserver_port=38385
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:34891
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:27.421687 3529 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:27.421962 3529 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:27.422128 3529 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:27.429271 3529 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:45:27.429373 3529 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:27.429419 3529 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:27.429525 3529 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:45:27.429605 3529 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:45:27.439400 3529 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:34891
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:34891
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=38385
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:27.440711 3529 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:27.442111 3529 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:27.454002 3537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:27.453857 3534 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:27.453879 3535 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:27.454685 3529 server_base.cc:1034] running on GCE node
I20260810 14:45:27.455535 3529 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:27.456915 3529 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:27.458132 3529 hybrid_clock.cc:648] HybridClock initialized: now 1786373127458098 us; error 44 us; skew 500 ppm
I20260810 14:45:27.458492 3529 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:27 dist-test-slave-vkjp krb5kdc[2902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373127, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:27.461687 3529 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:45:27.463248 3529 webserver.cc:460] Webserver started at http://127.24.215.190:38385/ using document root <none> and password file <none>
I20260810 14:45:27.463943 3529 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:27.464112 3529 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:27.469569 3529 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 14:45:27.472311 3545 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:27.473598 3529 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:45:27.473731 3529 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "707d39b966c04eae8ce44409e69fed0b"
format_stamp: "Formatted at 2026-08-10 14:45:24 on dist-test-slave-vkjp"
server_key: "d3c20c9436ee91f5acf3599e6ecee843"
server_key_iv: "d8a1c0098f46958a55be0f781b3797d7"
server_key_version: "encryptionkey@0"
I20260810 14:45:27.474638 3529 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:27.494953 3529 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:27.500487 3529 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:27.500720 3529 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:27.519513 3529 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:34891
I20260810 14:45:27.519537 3607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:34891 every 8 connection(s)
I20260810 14:45:27.521093 3529 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:45:27.523788 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 3529
I20260810 14:45:27.524767 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:43687
--local_ip_for_outbound_sockets=127.24.215.129
--tserver_master_addrs=127.24.215.190:34891
--webserver_port=44295
--webserver_interface=127.24.215.129
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.215.129
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:45:27.530491 3608 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:45:27.535656 3608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b: Bootstrap starting.
I20260810 14:45:27.554796 3608 log.cc:824] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:27.563644 3608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:45:27.564222 3608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b: Bootstrap complete.
I20260810 14:45:27.573362 3608 raft_consensus.cc:348] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707d39b966c04eae8ce44409e69fed0b" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34891 } }
I20260810 14:45:27.574186 3608 raft_consensus.cc:729] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707d39b966c04eae8ce44409e69fed0b, State: Initialized, Role: FOLLOWER
I20260810 14:45:27.574905 3608 consensus_queue.cc:260] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707d39b966c04eae8ce44409e69fed0b" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34891 } }
I20260810 14:45:27.575189 3608 raft_consensus.cc:388] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:45:27.575333 3608 raft_consensus.cc:482] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:45:27.575520 3608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:45:27.579782 3608 raft_consensus.cc:504] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707d39b966c04eae8ce44409e69fed0b" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34891 } }
I20260810 14:45:27.580302 3608 leader_election.cc:302] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 707d39b966c04eae8ce44409e69fed0b; no voters:
I20260810 14:45:27.580976 3608 leader_election.cc:288] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 14:45:27.581144 3614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 2 FOLLOWER]: Leader election won for term 2
I20260810 14:45:27.582410 3614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [term 2 LEADER]: Becoming Leader. State: Replica: 707d39b966c04eae8ce44409e69fed0b, State: Running, Role: LEADER
I20260810 14:45:27.583094 3614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707d39b966c04eae8ce44409e69fed0b" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34891 } }
I20260810 14:45:27.583411 3608 sys_catalog.cc:564] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:45:27.585659 3619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 707d39b966c04eae8ce44409e69fed0b. Latest consensus state: current_term: 2 leader_uuid: "707d39b966c04eae8ce44409e69fed0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707d39b966c04eae8ce44409e69fed0b" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34891 } } }
I20260810 14:45:27.586018 3619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:27.585764 3618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "707d39b966c04eae8ce44409e69fed0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707d39b966c04eae8ce44409e69fed0b" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34891 } } }
I20260810 14:45:27.586194 3618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:27.600395 3623 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:45:27.607967 3623 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:45:27.609916 3623 catalog_manager.cc:1259] Loaded cluster ID: eb482597a4494205a73ca99cc5c03f58
I20260810 14:45:27.610103 3623 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:45:27.614955 3623 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:45:27.617022 3623 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 707d39b966c04eae8ce44409e69fed0b: Loaded TSK: 0
I20260810 14:45:27.619062 3623 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 14:45:27.890733 3610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:27.891072 3610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:27.891278 3610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:27.898562 3610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:27.898723 3610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:27.898921 3610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:45:27.908969 3610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.129.keytab
--principal=oryx/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:43687
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=44295
--tserver_master_addrs=127.24.215.190:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:27.910429 3610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:27.911878 3610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:27.928295 3643 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:27.928319 3647 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:27.928423 3644 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:27.929232 3610 server_base.cc:1034] running on GCE node
I20260810 14:45:27.929992 3610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:27.931263 3610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:27.932516 3610 hybrid_clock.cc:648] HybridClock initialized: now 1786373127932439 us; error 61 us; skew 500 ppm
I20260810 14:45:27.932907 3610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:27 dist-test-slave-vkjp krb5kdc[2902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373127, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:27.940281 3610 init.cc:377] Logged in from keytab as oryx/127.24.215.129@KRBTEST.COM (short username oryx)
I20260810 14:45:27.942137 3610 webserver.cc:460] Webserver started at http://127.24.215.129:44295/ using document root <none> and password file <none>
I20260810 14:45:27.943030 3610 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:27.943204 3610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:27.950496 3610 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:45:27.954020 3655 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:27.955739 3610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 14:45:27.955945 3610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "18699830eb914492ad1a804f9a21bb13"
format_stamp: "Formatted at 2026-08-10 14:45:25 on dist-test-slave-vkjp"
server_key: "c5dde22fa130b7689caa815197302df0"
server_key_iv: "ec74e367233f8ebc79694c4671c050b0"
server_key_version: "encryptionkey@0"
I20260810 14:45:27.957358 3610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:27.977532 3610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:27.982672 3610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:27.982968 3610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:27.984089 3610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:27.985574 3610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:27.985703 3610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:27.985831 3610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:27.985919 3610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:28.018711 3610 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:43687
I20260810 14:45:28.018769 3796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:43687 every 8 connection(s)
I20260810 14:45:28.020239 3610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:45:28.025326 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 3610
I20260810 14:45:28.025979 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:43047
--local_ip_for_outbound_sockets=127.24.215.130
--tserver_master_addrs=127.24.215.190:34891
--webserver_port=37905
--webserver_interface=127.24.215.130
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.215.130
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 14:45:28.039376 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:28.023329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:46597 (local address 127.24.215.190:34891)
0810 14:45:28.024162 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:28.024180 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:28.024396 (+ 216us) server_negotiation.cc:407] Connection header received
0810 14:45:28.026688 (+ 2292us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:28.026700 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:28.027011 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:28.027413 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:28.028927 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.030137 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.031772 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.032418 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.032656 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:28.038417 (+ 5761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:46597: BlockingRecv error: recv got EOF from 127.24.215.129:46597 (error 108)
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":314,"threads_started":1}
I20260810 14:45:28.039448 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:28.023377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:46597)
0810 14:45:28.024214 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:28.024269 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:28.026094 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:28.027561 (+ 1467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:28.027574 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:28.028201 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:28.028723 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:28.028741 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.030309 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.030319 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:28.031614 (+ 1295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:28.031632 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.032694 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.032703 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:28.032981 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:28.033673 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:28.033696 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:28.038128 (+ 4432us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":539,"thread_start_us":331,"threads_started":1}
W20260810 14:45:28.040171 3797 heartbeater.cc:644] Failed to heartbeat to 127.24.215.190:34891 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.215.190:34891: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:28.049444 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:28.040791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:41711)
0810 14:45:28.040982 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:28.041001 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:28.041119 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:28.041508 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:28.041511 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:28.041801 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:28.041935 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:28.041942 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.043510 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.043514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:28.044383 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:28.044394 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.045470 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.045473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:28.045593 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:28.046204 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:28.046221 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:28.049141 (+ 2920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 14:45:28.049708 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:28.040967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:41711 (local address 127.24.215.190:34891)
0810 14:45:28.041185 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:28.041193 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:28.041216 (+ 23us) server_negotiation.cc:407] Connection header received
0810 14:45:28.041278 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:28.041285 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:28.041372 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:28.041480 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:28.042073 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.043375 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.044536 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.045311 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.045458 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:28.049422 (+ 3964us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:41711: BlockingRecv error: recv got EOF from 127.24.215.129:41711 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:28.058061 3801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:28.050236 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:36717)
0810 14:45:28.050387 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:28.050405 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:28.050496 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:28.050929 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:28.050936 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:28.051296 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:28.051437 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:28.051445 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.052408 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.052414 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:28.053318 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:28.053329 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.054037 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.054043 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:28.054247 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:28.054822 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:28.054839 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:28.057759 (+ 2920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 14:45:28.058351 3797 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:45:28.058521 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:28.050490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:36717 (local address 127.24.215.190:34891)
0810 14:45:28.050670 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:28.050675 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:28.050688 (+ 13us) server_negotiation.cc:407] Connection header received
0810 14:45:28.050728 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:28.050731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:28.050794 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:28.050893 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:28.051573 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.052261 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.053463 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.053890 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.054015 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:28.057976 (+ 3961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:36717: BlockingRecv error: recv got EOF from 127.24.215.129:36717 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 14:45:28.366529 3802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:28.366873 3802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:28.367048 3802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:28.374450 3802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:28.374612 3802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:28.374794 3802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:45:28.385234 3802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.130.keytab
--principal=oryx/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:43047
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=37905
--tserver_master_addrs=127.24.215.190:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:28.386662 3802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:28.388082 3802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:28.401615 3811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:28.401573 3813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:28.401824 3810 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:28.402362 3802 server_base.cc:1034] running on GCE node
I20260810 14:45:28.403029 3802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:28.404099 3802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:28.405333 3802 hybrid_clock.cc:648] HybridClock initialized: now 1786373128405290 us; error 55 us; skew 500 ppm
I20260810 14:45:28.405663 3802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373128, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:28.408989 3802 init.cc:377] Logged in from keytab as oryx/127.24.215.130@KRBTEST.COM (short username oryx)
I20260810 14:45:28.410495 3802 webserver.cc:460] Webserver started at http://127.24.215.130:37905/ using document root <none> and password file <none>
I20260810 14:45:28.411263 3802 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:28.411389 3802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:28.417040 3802 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 14:45:28.419824 3821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:28.421223 3802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:45:28.421373 3802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "0e69c71ec977486897afec01b1878ea5"
format_stamp: "Formatted at 2026-08-10 14:45:25 on dist-test-slave-vkjp"
server_key: "74c79f524aca207036dfc7dad4669219"
server_key_iv: "dcd774a3761833fc5c1bddef36713175"
server_key_version: "encryptionkey@0"
I20260810 14:45:28.422375 3802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:28.454123 3802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:28.458840 3802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:28.459216 3802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:28.460430 3802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:28.461905 3802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:28.462023 3802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:28.462157 3802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:28.462250 3802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:28.496568 3802 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:43047
I20260810 14:45:28.496670 3956 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:43047 every 8 connection(s)
I20260810 14:45:28.498097 3802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:45:28.505795 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 3802
I20260810 14:45:28.506561 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:44287
--local_ip_for_outbound_sockets=127.24.215.131
--tserver_master_addrs=127.24.215.190:34891
--webserver_port=34811
--webserver_interface=127.24.215.131
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:38869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.24.215.131
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 14:45:28.517738 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:28.500979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:55163 (local address 127.24.215.190:34891)
0810 14:45:28.501287 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:28.501294 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:28.503061 (+ 1767us) server_negotiation.cc:407] Connection header received
0810 14:45:28.504992 (+ 1931us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:28.504999 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:28.505089 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:28.505186 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:28.507433 (+ 2247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.508157 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.510730 (+ 2573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.511501 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.511625 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:28.517479 (+ 5854us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:55163: BlockingRecv error: recv got EOF from 127.24.215.130:55163 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
I20260810 14:45:28.518550 3959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:28.501735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:55163)
0810 14:45:28.502842 (+ 1107us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:28.502908 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:28.504693 (+ 1785us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:28.505480 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:28.505493 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:28.506322 (+ 829us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:28.507225 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:28.507256 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.508337 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.508345 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:28.510535 (+ 2190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:28.510554 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.511997 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.512003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:28.512213 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:28.513050 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:28.513084 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:28.517283 (+ 4199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":443,"threads_started":1}
W20260810 14:45:28.519410 3957 heartbeater.cc:644] Failed to heartbeat to 127.24.215.190:34891 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.215.190:34891: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:28.531376 3959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:28.520124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:34175)
0810 14:45:28.520392 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:28.520411 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:28.520542 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:28.521255 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:28.521260 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:28.521696 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:28.521874 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:28.521885 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.523575 (+ 1690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.523581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:28.525036 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:28.525052 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.526198 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.526204 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:28.526523 (+ 319us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:28.527231 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:28.527255 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:28.531054 (+ 3799us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139}
W20260810 14:45:28.532080 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:28.520245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:34175 (local address 127.24.215.190:34891)
0810 14:45:28.520873 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:28.520881 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:28.520904 (+ 23us) server_negotiation.cc:407] Connection header received
0810 14:45:28.520973 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:28.520981 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:28.521081 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:28.521195 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:28.522081 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.523407 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.525198 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.526025 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.526163 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:28.531416 (+ 5253us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:34175: BlockingRecv error: recv got EOF from 127.24.215.130:34175 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:28.541975 3959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:28.532287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:51427)
0810 14:45:28.532847 (+ 560us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:28.532872 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:28.533008 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:28.533336 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:28.533344 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:28.533821 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:28.534065 (+ 244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:28.534081 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.535079 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.535110 (+ 31us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:28.536879 (+ 1769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:28.536896 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.537526 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.537532 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:28.537715 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:28.538362 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:28.538385 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:28.541671 (+ 3286us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421}
W20260810 14:45:28.542062 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:28.532633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:51427 (local address 127.24.215.190:34891)
0810 14:45:28.532802 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:28.532807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:28.532945 (+ 138us) server_negotiation.cc:407] Connection header received
0810 14:45:28.533129 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:28.533133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:28.533188 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:28.533266 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:28.534225 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.534931 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.537020 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.537390 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.537490 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:28.541871 (+ 4381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:51427: BlockingRecv error: recv got EOF from 127.24.215.130:51427 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260810 14:45:28.542315 3957 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:45:28.853706 3960 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:28.854020 3960 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:28.854130 3960 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:28.861519 3960 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:28.861641 3960 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:28.861755 3960 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:45:28.871881 3960 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:38869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/krb5kdc/oryx_127.24.215.131.keytab
--principal=oryx/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:44287
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=34811
--tserver_master_addrs=127.24.215.190:34891
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:28.873167 3960 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:28.874420 3960 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:28.888517 3966 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:28.888628 3965 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:28.888602 3968 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:28.889343 3960 server_base.cc:1034] running on GCE node
I20260810 14:45:28.889955 3960 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:28.890946 3960 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:28.892148 3960 hybrid_clock.cc:648] HybridClock initialized: now 1786373128892099 us; error 86 us; skew 500 ppm
I20260810 14:45:28.892462 3960 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373128, etypes {rep=17 tkt=17 ses=17}, oryx/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:28.900900 3960 init.cc:377] Logged in from keytab as oryx/127.24.215.131@KRBTEST.COM (short username oryx)
I20260810 14:45:28.902426 3960 webserver.cc:460] Webserver started at http://127.24.215.131:34811/ using document root <none> and password file <none>
I20260810 14:45:28.903493 3960 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:28.903676 3960 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:28.909662 3960 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 14:45:28.912413 3975 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:28.913854 3960 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:45:28.914033 3960 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "5f0efa1d27f344c7addd3f06affc103d"
format_stamp: "Formatted at 2026-08-10 14:45:26 on dist-test-slave-vkjp"
server_key: "f5169af1d45bb6a6dd2e16fc18da12ba"
server_key_iv: "c35cedb93f6b0d002971cfa73c5bca50"
server_key_version: "encryptionkey@0"
I20260810 14:45:28.914981 3960 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:28.929899 3960 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:28.933979 3960 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:28.934218 3960 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:28.935263 3960 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:28.936700 3960 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:28.936789 3960 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:28.936862 3960 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:28.936919 3960 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:28.970642 3960 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:44287
I20260810 14:45:28.970721 4112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:44287 every 8 connection(s)
I20260810 14:45:28.972209 3960 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:45:28.977411 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 3960
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 14:45:28.990712 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:28.974803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:51349 (local address 127.24.215.190:34891)
0810 14:45:28.975007 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:28.975012 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:28.976159 (+ 1147us) server_negotiation.cc:407] Connection header received
0810 14:45:28.978197 (+ 2038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:28.978203 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:28.978297 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:28.978412 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:28.980805 (+ 2393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.981730 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.983823 (+ 2093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.984446 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.984586 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:28.990419 (+ 5833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:51349: BlockingRecv error: recv got EOF from 127.24.215.131:51349 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 14:45:28.991619 4118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:28.975302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:51349)
0810 14:45:28.975992 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:28.976029 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:28.977799 (+ 1770us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:28.978782 (+ 983us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:28.978798 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:28.979658 (+ 860us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:28.980560 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:28.980588 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.981916 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.981920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:28.983668 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:28.983684 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.984655 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.984660 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:28.984857 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:28.985664 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:28.985689 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:28.990191 (+ 4502us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":254,"threads_started":1}
W20260810 14:45:28.992317 4113 heartbeater.cc:644] Failed to heartbeat to 127.24.215.190:34891 (0 consecutive failures): Not authorized: Failed to ping master at 127.24.215.190:34891: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
I20260810 14:45:28.995442 4120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:28.978581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46100 (local address 127.24.215.190:34891)
0810 14:45:28.979414 (+ 833us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:28.979418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:28.979435 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:45:28.979549 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:28.979552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:28.979606 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:28.979691 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:28.981605 (+ 1914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.982521 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.984335 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.984916 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.985106 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:28.988217 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:28.988226 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:28.988245 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:28.988288 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:28.991601 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:28.992484 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:28.992496 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:28.992504 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:28.992722 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:28.993348 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:28.993353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:28.993356 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:28.994038 (+ 682us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:28.994390 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:28.994730 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 14:45:28.995060 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":612,"threads_started":1}
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:28 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:29.002221 4118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:28.993014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:42999)
0810 14:45:28.993211 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:28.993230 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:28.993340 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:28.993664 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:28.993669 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:28.994093 (+ 424us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:28.994280 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:28.994294 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.995650 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.995655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:28.996580 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:28.996593 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.997357 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.997363 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:28.997577 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:28.998347 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:28.998376 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:29.001909 (+ 3533us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 14:45:29.002496 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:28.993100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:42999 (local address 127.24.215.190:34891)
0810 14:45:28.993269 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:28.993274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:28.993292 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:45:28.993470 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:28.993475 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:28.993530 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:28.993602 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:28.994435 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.995447 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.996733 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:28.997198 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:28.997337 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:29.002194 (+ 4857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:42999: BlockingRecv error: recv got EOF from 127.24.215.131:42999 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 14:45:29 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:29 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:29.014395 4118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:29.003206 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:59411)
0810 14:45:29.003415 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:29.003446 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:29.003553 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:29.003895 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:29.003900 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:29.004304 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:29.004531 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:29.004547 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:29.005896 (+ 1349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:29.005902 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:29.007444 (+ 1542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:29.007463 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:29.008400 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:29.008409 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:29.008659 (+ 250us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:29.009421 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:29.009445 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:29.014048 (+ 4603us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 14:45:29.014578 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:29.003294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:59411 (local address 127.24.215.190:34891)
0810 14:45:29.003517 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:29.003520 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:29.003537 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:45:29.003664 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:29.003669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:29.003743 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:29.003846 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:29.004684 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:29.005458 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:29.007603 (+ 2145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:29.008204 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:29.008367 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:29.014288 (+ 5921us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:59411: BlockingRecv error: recv got EOF from 127.24.215.131:59411 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
W20260810 14:45:29.014791 4113 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 10 14:45:29 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:29 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:29.068347 4121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:29.059191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:45191)
0810 14:45:29.059752 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:29.059767 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:29.059933 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:29.060462 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:29.060467 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:29.061123 (+ 656us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:29.061371 (+ 248us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:29.061381 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:29.062550 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:29.062555 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:29.063725 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:29.063738 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:29.064499 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:29.064503 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:29.064622 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:29.065259 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:29.065277 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:29.068120 (+ 2843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":448,"thread_start_us":332,"threads_started":1}
W20260810 14:45:29.068635 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:29.059206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:45191 (local address 127.24.215.190:34891)
0810 14:45:29.059392 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:29.059397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:29.059857 (+ 460us) server_negotiation.cc:407] Connection header received
0810 14:45:29.060101 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:29.060106 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:29.060215 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:29.060431 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:29.061553 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:29.062371 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:29.063856 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:29.064325 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:29.064434 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:29.068351 (+ 3917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:45191: BlockingRecv error: recv got EOF from 127.24.215.129:45191 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 10 14:45:29 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:29 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:29.551884 4122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:29.542917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:54725)
0810 14:45:29.543481 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:29.543498 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:29.543664 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:29.544024 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:29.544031 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:29.544538 (+ 507us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:29.544728 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:29.544737 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:29.545781 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:29.545787 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:29.546756 (+ 969us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:29.546767 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:29.547606 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:29.547611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:29.547781 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:29.548430 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:29.548448 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:29.551550 (+ 3102us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":295,"threads_started":1}
W20260810 14:45:29.552016 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:29.542993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:54725 (local address 127.24.215.190:34891)
0810 14:45:29.543285 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:29.543292 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:29.543587 (+ 295us) server_negotiation.cc:407] Connection header received
0810 14:45:29.543796 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:29.543800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:29.543862 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:29.543981 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:29.544881 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:29.545606 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:29.546903 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:29.547416 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:29.547547 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:29.551776 (+ 4229us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:54725: BlockingRecv error: recv got EOF from 127.24.215.130:54725 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 10 14:45:30 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:30 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:30.024693 4123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:30.015480 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:35987)
0810 14:45:30.015989 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:30.016004 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:30.016164 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:30.016516 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:30.016521 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:30.017022 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:30.017211 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:30.017220 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:30.018246 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:30.018252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:30.019861 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:30.019874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:30.020786 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:30.020790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:30.020907 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:30.021577 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:30.021594 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:30.024455 (+ 2861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":274,"threads_started":1}
W20260810 14:45:30.024845 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:30.015539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:35987 (local address 127.24.215.190:34891)
0810 14:45:30.015762 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:30.015771 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:30.016092 (+ 321us) server_negotiation.cc:407] Connection header received
0810 14:45:30.016290 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:30.016295 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:30.016369 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:30.016472 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:30.017361 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:30.018095 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:30.020003 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:30.020615 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:30.020738 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:30.024647 (+ 3909us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:35987: BlockingRecv error: recv got EOF from 127.24.215.131:35987 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 14:45:30 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:30 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:30.077733 4124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:30.069259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:34641)
0810 14:45:30.069777 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:30.069793 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:30.069947 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:30.070330 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:30.070335 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:30.070756 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:30.070943 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:30.070952 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:30.072068 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:30.072077 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:30.072979 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:30.072990 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:30.073708 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:30.073714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:30.073891 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:30.074563 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:30.074580 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:30.077352 (+ 2772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":296,"threads_started":1}
W20260810 14:45:30.077867 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:30.069347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:34641 (local address 127.24.215.190:34891)
0810 14:45:30.069559 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:30.069567 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:30.069873 (+ 306us) server_negotiation.cc:407] Connection header received
0810 14:45:30.070103 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:30.070109 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:30.070193 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:30.070293 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:30.071178 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:30.071923 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:30.073117 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:30.073551 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:30.073683 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:30.077622 (+ 3939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:34641: BlockingRecv error: recv got EOF from 127.24.215.129:34641 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 14:45:30 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:30 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:30.562808 4125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:30.552854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:51543)
0810 14:45:30.553514 (+ 660us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:30.553532 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:30.553725 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:30.554157 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:30.554163 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:30.554667 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:30.554889 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:30.554902 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:30.556154 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:30.556161 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:30.557238 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:30.557252 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:30.558126 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:30.558133 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:30.558344 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:30.559189 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:30.559215 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:30.562508 (+ 3293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":518,"thread_start_us":345,"threads_started":1}
W20260810 14:45:30.562870 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:30.553006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:51543 (local address 127.24.215.190:34891)
0810 14:45:30.553346 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:30.553354 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:30.553641 (+ 287us) server_negotiation.cc:407] Connection header received
0810 14:45:30.553897 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:30.553904 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:30.553990 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:30.554116 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:30.555111 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:30.555989 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:30.557417 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:30.557933 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:30.558139 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:30.562696 (+ 4557us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:51543: BlockingRecv error: recv got EOF from 127.24.215.130:51543 (error 108)
Metrics: {"server-negotiator.queue_time_us":98}
Aug 10 14:45:31 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:31 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:31.034714 4127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:31.025749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:58705)
0810 14:45:31.026359 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:31.026376 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:31.026493 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:31.026862 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:31.026866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:31.027420 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:31.027592 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:31.027602 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:31.028743 (+ 1141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:31.028748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:31.029836 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:31.029854 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:31.030848 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:31.030854 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:31.031003 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:31.031710 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:31.031728 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:31.034468 (+ 2740us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":293,"threads_started":1}
W20260810 14:45:31.034932 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:31.025947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:58705 (local address 127.24.215.190:34891)
0810 14:45:31.026118 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:31.026123 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:31.026454 (+ 331us) server_negotiation.cc:407] Connection header received
0810 14:45:31.026652 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:31.026657 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:31.026724 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:31.026820 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:31.027747 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:31.028546 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:31.030003 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:31.030651 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:31.030800 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:31.034664 (+ 3864us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:58705: BlockingRecv error: recv got EOF from 127.24.215.131:58705 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 14:45:31 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:31 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:31.086845 4128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:31.078712 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:57875)
0810 14:45:31.079361 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:31.079376 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:31.079495 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:31.079790 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:31.079793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:31.080148 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:31.080320 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:31.080328 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:31.081400 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:31.081404 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:31.082291 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:31.082301 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:31.082897 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:31.082900 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:31.083025 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:31.083626 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:31.083643 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:31.086601 (+ 2958us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":411,"threads_started":1}
W20260810 14:45:31.087029 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:31.078772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:57875 (local address 127.24.215.190:34891)
0810 14:45:31.078959 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:31.078964 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:31.079458 (+ 494us) server_negotiation.cc:407] Connection header received
0810 14:45:31.079599 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:31.079603 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:31.079667 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:31.079750 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:31.080448 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:31.081253 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:31.082403 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:31.082783 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:31.082870 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:31.086776 (+ 3906us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:57875: BlockingRecv error: recv got EOF from 127.24.215.129:57875 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 14:45:31 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:31 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:31.571923 4129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:31.563791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:40055)
0810 14:45:31.564281 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:31.564298 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:31.564411 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:31.564714 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:31.564717 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:31.565059 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:31.565225 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:31.565234 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:31.566220 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:31.566224 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:31.567190 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:31.567203 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:31.567844 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:31.567848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:31.567974 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:31.568632 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:31.568651 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:31.571530 (+ 2879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":262,"threads_started":1}
W20260810 14:45:31.571998 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:31.563908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:40055 (local address 127.24.215.190:34891)
0810 14:45:31.564145 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:31.564152 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:31.564377 (+ 225us) server_negotiation.cc:407] Connection header received
0810 14:45:31.564523 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:31.564527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:31.564590 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:31.564669 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:31.565350 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:31.566112 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:31.567307 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:31.567724 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:31.567814 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:31.571733 (+ 3919us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:40055: BlockingRecv error: recv got EOF from 127.24.215.130:40055 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 10 14:45:32 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:32 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:32.044180 4130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:32.035667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:36173)
0810 14:45:32.036214 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:32.036230 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:32.036345 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:32.036657 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:32.036659 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:32.037031 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:32.037234 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:32.037244 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:32.038166 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:32.038169 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:32.039045 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:32.039056 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:32.039784 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:32.039790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:32.040013 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:32.040740 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:32.040764 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:32.043775 (+ 3011us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":303,"threads_started":1}
W20260810 14:45:32.044234 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:32.035899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:36173 (local address 127.24.215.190:34891)
0810 14:45:32.036070 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:32.036074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:32.036308 (+ 234us) server_negotiation.cc:407] Connection header received
0810 14:45:32.036456 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:32.036459 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:32.036529 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:32.036611 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:32.037363 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:32.038050 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:32.039199 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:32.039619 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:32.039761 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:32.044000 (+ 4239us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:36173: BlockingRecv error: recv got EOF from 127.24.215.131:36173 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 14:45:32 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:32 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:32.096251 4131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:32.087826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:55691)
0810 14:45:32.088385 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:32.088406 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:32.088557 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:32.088909 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:32.088913 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:32.089300 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:32.089442 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:32.089450 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:32.090448 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:32.090454 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:32.091578 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:32.091591 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:32.092280 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:32.092286 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:32.092460 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:32.093128 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:32.093146 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:32.095900 (+ 2754us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":311,"threads_started":1}
W20260810 14:45:32.096379 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:32.087856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:55691 (local address 127.24.215.190:34891)
0810 14:45:32.088090 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:32.088099 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:32.088498 (+ 399us) server_negotiation.cc:407] Connection header received
0810 14:45:32.088704 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:32.088715 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:32.088776 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:32.088875 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:32.089579 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:32.090316 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:32.091721 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:32.092126 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:32.092257 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:32.096122 (+ 3865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:55691: BlockingRecv error: recv got EOF from 127.24.215.129:55691 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 10 14:45:32 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:32 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:32.582120 4132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:32.573207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:57831)
0810 14:45:32.573722 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:32.573738 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:32.573895 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:32.574283 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:32.574288 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:32.574757 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:32.574959 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:32.574988 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:32.576341 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:32.576347 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:32.577439 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:32.577452 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:32.578204 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:32.578208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:32.578337 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:32.578998 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:32.579017 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:32.581885 (+ 2868us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":420,"thread_start_us":291,"threads_started":1}
W20260810 14:45:32.582273 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:32.573263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:57831 (local address 127.24.215.190:34891)
0810 14:45:32.573494 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:32.573502 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:32.573824 (+ 322us) server_negotiation.cc:407] Connection header received
0810 14:45:32.574049 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:32.574060 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:32.574138 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:32.574246 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:32.575152 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:32.576168 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:32.577579 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:32.578047 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:32.578140 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:32.582063 (+ 3923us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:57831: BlockingRecv error: recv got EOF from 127.24.215.130:57831 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 14:45:33 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:33 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:33.056614 4134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:33.045195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:47619)
0810 14:45:33.045715 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:33.045732 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:33.045845 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:33.046135 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:33.046138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:33.046512 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:33.046662 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:33.046671 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:33.047688 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:33.047692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:33.048666 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:33.048678 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:33.049299 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:33.049302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:33.049423 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:33.053114 (+ 3691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:33.053132 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:33.056377 (+ 3245us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":300,"threads_started":1}
W20260810 14:45:33.056805 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:33.045237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:47619 (local address 127.24.215.190:34891)
0810 14:45:33.045408 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:33.045412 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:33.045805 (+ 393us) server_negotiation.cc:407] Connection header received
0810 14:45:33.045943 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:33.045946 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:33.046007 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:33.046085 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:33.046798 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:33.047579 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:33.048788 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:33.049182 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:33.049269 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:33.056570 (+ 7301us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:47619: BlockingRecv error: recv got EOF from 127.24.215.131:47619 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 10 14:45:33 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:33 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:33.105679 4135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:33.097466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:56219)
0810 14:45:33.098062 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:33.098079 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:33.098192 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:33.098487 (+ 295us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:33.098491 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:33.098882 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:33.099040 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:33.099048 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:33.100063 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:33.100067 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:33.100971 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:33.100982 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:33.101619 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:33.101623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:33.101745 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:33.102306 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:33.102333 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:33.105436 (+ 3103us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":300,"threads_started":1}
W20260810 14:45:33.105836 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:33.097562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:56219 (local address 127.24.215.190:34891)
0810 14:45:33.097748 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:33.097753 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:33.098148 (+ 395us) server_negotiation.cc:407] Connection header received
0810 14:45:33.098291 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:33.098295 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:33.098352 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:33.098433 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:33.099204 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:33.099943 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:33.101107 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:33.101501 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:33.101591 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:33.105620 (+ 4029us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:56219: BlockingRecv error: recv got EOF from 127.24.215.129:56219 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 10 14:45:33 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:33 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
W20260810 14:45:33.593323 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:33.583374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:36791 (local address 127.24.215.190:34891)
0810 14:45:33.583591 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:33.583597 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:33.583812 (+ 215us) server_negotiation.cc:407] Connection header received
0810 14:45:33.583951 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:33.583954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:33.584018 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:33.584095 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:33.584893 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:33.585647 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:33.587073 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:33.587924 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:33.588087 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:33.593026 (+ 4939us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:36791: BlockingRecv error: recv got EOF from 127.24.215.130:36791 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 14:45:33.593540 4136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:33.583149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:36791)
0810 14:45:33.583742 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:33.583759 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:33.583870 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:33.584110 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:33.584113 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:33.584506 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:33.584725 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:33.584736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:33.585767 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:33.585770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:33.586866 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:33.586881 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:33.588107 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:33.588112 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:33.588325 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:33.589160 (+ 835us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:33.589185 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:33.592824 (+ 3639us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":271,"threads_started":1}
Aug 10 14:45:34 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:34 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:34.066275 4137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:34.057737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:60375)
0810 14:45:34.058376 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:34.058402 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:34.058547 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:34.058884 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:34.058889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:34.059363 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:34.059523 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:34.059532 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:34.060567 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:34.060576 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:34.061523 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:34.061535 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:34.062276 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:34.062282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:34.062471 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:34.063034 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:34.063069 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:34.065877 (+ 2808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":490,"thread_start_us":280,"threads_started":1}
W20260810 14:45:34.066370 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:34.057804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:60375 (local address 127.24.215.190:34891)
0810 14:45:34.058016 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:34.058024 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:34.058503 (+ 479us) server_negotiation.cc:407] Connection header received
0810 14:45:34.058671 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:34.058676 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:34.058745 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:34.058848 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:34.059680 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:34.060402 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:34.061670 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:34.062101 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:34.062251 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:34.066098 (+ 3847us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:60375: BlockingRecv error: recv got EOF from 127.24.215.131:60375 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 10 14:45:34 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:34 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:34.114825 4139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:34.106683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:48501)
0810 14:45:34.107246 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:34.107262 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:34.107374 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:34.107682 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:34.107686 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:34.108082 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:34.108240 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:34.108248 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:34.109259 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:34.109262 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:34.110193 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:34.110204 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:34.110860 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:34.110864 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:34.110982 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:34.111666 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:34.111683 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:34.114618 (+ 2935us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":291,"threads_started":1}
W20260810 14:45:34.115073 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:34.106731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:48501 (local address 127.24.215.190:34891)
0810 14:45:34.106916 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:34.106921 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:34.107333 (+ 412us) server_negotiation.cc:407] Connection header received
0810 14:45:34.107484 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:34.107487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:34.107546 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:34.107635 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:34.108356 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:34.109144 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:34.110318 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:34.110729 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:34.110820 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:34.114797 (+ 3977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:48501: BlockingRecv error: recv got EOF from 127.24.215.129:48501 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 14:45:34 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:34 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:34.602685 4141 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:34.594809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:40549)
0810 14:45:34.595441 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:34.595457 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:34.595571 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:34.595867 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:34.595871 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:34.596258 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:34.596415 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:34.596437 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:34.597376 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:34.597380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:34.598270 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:34.598281 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:34.598935 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:34.598939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:34.599061 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:34.599716 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:34.599734 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:34.602466 (+ 2732us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":387,"threads_started":1}
W20260810 14:45:34.602994 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:34.594858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:40549 (local address 127.24.215.190:34891)
0810 14:45:34.595070 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:34.595074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:34.595541 (+ 467us) server_negotiation.cc:407] Connection header received
0810 14:45:34.595674 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:34.595678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:34.595741 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:34.595820 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:34.596553 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:34.597254 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:34.598408 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:34.598800 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:34.598893 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:34.602690 (+ 3797us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:40549: BlockingRecv error: recv got EOF from 127.24.215.130:40549 (error 108)
Metrics: {"server-negotiator.queue_time_us":91}
Aug 10 14:45:35 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:35 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:35.075835 4142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:35.067374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:33615)
0810 14:45:35.067887 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:35.067904 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:35.068030 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:35.068331 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:35.068334 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:35.068756 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:35.068918 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:35.068927 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:35.069969 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:35.069973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:35.070912 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:35.070926 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:35.071738 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:35.071743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:35.071868 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:35.072553 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:35.072571 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:35.075615 (+ 3044us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":294,"threads_started":1}
W20260810 14:45:35.076036 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:35.067481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:33615 (local address 127.24.215.190:34891)
0810 14:45:35.067753 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:35.067759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:35.068003 (+ 244us) server_negotiation.cc:407] Connection header received
0810 14:45:35.068135 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:35.068139 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:35.068202 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:35.068276 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:35.069049 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:35.069818 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:35.071051 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:35.071597 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:35.071693 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:35.075808 (+ 4115us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:33615: BlockingRecv error: recv got EOF from 127.24.215.131:33615 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 10 14:45:35 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:35 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:35.124567 4143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:35.115770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:45867)
0810 14:45:35.116416 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:35.116441 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:35.116566 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:35.116853 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:35.116857 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:35.117323 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:35.117500 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:35.117510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:35.118554 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:35.118558 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:35.119634 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:35.119647 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:35.120290 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:35.120294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:35.120463 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:35.121185 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:35.121208 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:35.124307 (+ 3099us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":283,"threads_started":1}
W20260810 14:45:35.124703 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:35.115879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:45867 (local address 127.24.215.190:34891)
0810 14:45:35.116130 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:35.116139 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:35.116509 (+ 370us) server_negotiation.cc:407] Connection header received
0810 14:45:35.116668 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:35.116672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:35.116734 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:35.116813 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:35.117627 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:35.118413 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:35.119756 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:35.120162 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:35.120256 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:35.124502 (+ 4246us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:45867: BlockingRecv error: recv got EOF from 127.24.215.129:45867 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 10 14:45:35 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:35 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:35.612833 4144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:35.603786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:39945)
0810 14:45:35.604443 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:35.604466 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:35.604628 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:35.605041 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:35.605048 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:35.605694 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:35.605946 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:35.605960 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:35.607339 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:35.607344 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:35.608276 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:35.608289 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:35.608989 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:35.608997 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:35.609117 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:35.609692 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:35.609710 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:35.612590 (+ 2880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":291,"threads_started":1}
W20260810 14:45:35.613166 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:35.603929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:39945 (local address 127.24.215.190:34891)
0810 14:45:35.604217 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:35.604223 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:35.604599 (+ 376us) server_negotiation.cc:407] Connection header received
0810 14:45:35.604786 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:35.604795 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:35.604890 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:35.604975 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:35.606174 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:35.607184 (+ 1010us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:35.608420 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:35.608839 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:35.608946 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:35.612825 (+ 3879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:39945: BlockingRecv error: recv got EOF from 127.24.215.130:39945 (error 108)
Metrics: {"server-negotiator.queue_time_us":124}
Aug 10 14:45:36 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:36 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:36.085589 4145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:36.076814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:59735)
0810 14:45:36.077430 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:36.077446 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:36.077602 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:36.077986 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:36.077991 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:36.078443 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:36.078608 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:36.078617 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:36.079703 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:36.079711 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:36.080648 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:36.080659 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:36.081406 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:36.081412 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:36.081580 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:36.082206 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:36.082223 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:36.085292 (+ 3069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":290,"threads_started":1}
W20260810 14:45:36.085750 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:36.076925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:59735 (local address 127.24.215.190:34891)
0810 14:45:36.077111 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:36.077120 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:36.077549 (+ 429us) server_negotiation.cc:407] Connection header received
0810 14:45:36.077746 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:36.077750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:36.077814 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:36.077933 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:36.078751 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:36.079564 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:36.080790 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:36.081246 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:36.081383 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:36.085529 (+ 4146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:59735: BlockingRecv error: recv got EOF from 127.24.215.131:59735 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 10 14:45:36 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:36 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:36.134004 4146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:36.125601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:53867)
0810 14:45:36.126134 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:36.126150 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:36.126261 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:36.126669 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:36.126674 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:36.127070 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:36.127254 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:36.127263 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:36.128344 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:36.128348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:36.129274 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:36.129285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:36.130078 (+ 793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:36.130082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:36.130188 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:36.130823 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:36.130845 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:36.133773 (+ 2928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":262,"threads_started":1}
W20260810 14:45:36.134338 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:36.125843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:53867 (local address 127.24.215.190:34891)
0810 14:45:36.126047 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:36.126052 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:36.126221 (+ 169us) server_negotiation.cc:407] Connection header received
0810 14:45:36.126421 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:36.126424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:36.126509 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:36.126592 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:36.127406 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:36.128189 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:36.129415 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:36.129904 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:36.130012 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:36.133996 (+ 3984us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:53867: BlockingRecv error: recv got EOF from 127.24.215.129:53867 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 10 14:45:36 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:36 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:36.622752 4147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:36.613859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:56853)
0810 14:45:36.614506 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:36.614526 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:36.614676 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:36.615075 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:36.615080 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:36.615537 (+ 457us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:36.615746 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:36.615764 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:36.616923 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:36.616928 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:36.617924 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:36.617944 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:36.618703 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:36.618708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:36.618830 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:36.619538 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:36.619556 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:36.622414 (+ 2858us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":373,"threads_started":1}
W20260810 14:45:36.622874 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:36.614014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:56853 (local address 127.24.215.190:34891)
0810 14:45:36.614228 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:36.614235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:36.614589 (+ 354us) server_negotiation.cc:407] Connection header received
0810 14:45:36.614816 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:36.614821 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:36.614902 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:36.615012 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:36.615889 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:36.616763 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:36.618079 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:36.618510 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:36.618603 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:36.622649 (+ 4046us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:56853: BlockingRecv error: recv got EOF from 127.24.215.130:56853 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 10 14:45:37 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:37 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:37.094666 4149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:37.086526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:54231)
0810 14:45:37.087046 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:37.087064 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:37.087348 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:37.087713 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:37.087716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:37.088120 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:37.088266 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:37.088293 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:37.089287 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:37.089290 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:37.090211 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:37.090221 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:37.090929 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:37.090933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:37.091040 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:37.091649 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:37.091666 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:37.094434 (+ 2768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":278,"threads_started":1}
W20260810 14:45:37.094897 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:37.086593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:54231 (local address 127.24.215.190:34891)
0810 14:45:37.086840 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:37.086846 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:37.087218 (+ 372us) server_negotiation.cc:407] Connection header received
0810 14:45:37.087491 (+ 273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:37.087494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:37.087559 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:37.087636 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:37.088417 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:37.089153 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:37.090373 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:37.090770 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:37.090862 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:37.094625 (+ 3763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:54231: BlockingRecv error: recv got EOF from 127.24.215.131:54231 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 10 14:45:37 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:37 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:37.143014 4150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:37.134843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:45903)
0810 14:45:37.135371 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:37.135388 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:37.135502 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:37.135818 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:37.135821 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:37.136226 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:37.136434 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:37.136444 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:37.137448 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:37.137452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:37.138399 (+ 947us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:37.138410 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:37.139116 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:37.139126 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:37.139260 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:37.139903 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:37.139920 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:37.142777 (+ 2857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":312,"threads_started":1}
W20260810 14:45:37.143292 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:37.134943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:45903 (local address 127.24.215.190:34891)
0810 14:45:37.135222 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:37.135228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:37.135465 (+ 237us) server_negotiation.cc:407] Connection header received
0810 14:45:37.135611 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:37.135615 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:37.135694 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:37.135773 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:37.136577 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:37.137336 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:37.138523 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:37.138941 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:37.139033 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:37.142998 (+ 3965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:45903: BlockingRecv error: recv got EOF from 127.24.215.129:45903 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 10 14:45:37 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:37 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:37.632627 4152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:37.624054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:56421)
0810 14:45:37.624628 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:37.624646 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:37.624761 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:37.625036 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:37.625038 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:37.625539 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:37.625766 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:37.625776 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:37.626956 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:37.626960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:37.628003 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:37.628017 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:37.628747 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:37.628752 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:37.628884 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:37.629452 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:37.629471 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:37.632383 (+ 2912us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":318,"threads_started":1}
W20260810 14:45:37.632815 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:37.624082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:56421 (local address 127.24.215.190:34891)
0810 14:45:37.624244 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:37.624249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:37.624715 (+ 466us) server_negotiation.cc:407] Connection header received
0810 14:45:37.624873 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:37.624877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:37.624939 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:37.625017 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:37.625912 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:37.626807 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:37.628161 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:37.628593 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:37.628696 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:37.632593 (+ 3897us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:56421: BlockingRecv error: recv got EOF from 127.24.215.130:56421 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 10 14:45:38 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:38 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:38.108597 4154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:38.095702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:49305)
0810 14:45:38.096248 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:38.096264 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:38.096431 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:38.096807 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:38.096812 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:38.097320 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:38.097521 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:38.097531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:38.098725 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:38.098730 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:38.099705 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:38.099716 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:38.100498 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:38.100503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:38.100694 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:38.105042 (+ 4348us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:38.105062 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:38.108178 (+ 3116us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":322,"threads_started":1}
W20260810 14:45:38.108716 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:38.095758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:49305 (local address 127.24.215.190:34891)
0810 14:45:38.096030 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:38.096037 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:38.096367 (+ 330us) server_negotiation.cc:407] Connection header received
0810 14:45:38.096602 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:38.096606 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:38.096667 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:38.096774 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:38.097671 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:38.098592 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:38.099870 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:38.100320 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:38.100473 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:38.108451 (+ 7978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:49305: BlockingRecv error: recv got EOF from 127.24.215.131:49305 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 10 14:45:38 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:38 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:38.152783 4155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:38.143955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:35145)
0810 14:45:38.144482 (+ 527us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:38.144501 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:38.144647 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:38.144986 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:38.144990 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:38.145453 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:38.145643 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:38.145653 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:38.146854 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:38.146862 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:38.147964 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:38.147976 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:38.148741 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:38.148747 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:38.148929 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:38.149580 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:38.149599 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:38.152551 (+ 2952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":310,"threads_started":1}
W20260810 14:45:38.152962 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:38.144073 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:35145 (local address 127.24.215.190:34891)
0810 14:45:38.144250 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:38.144255 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:38.144585 (+ 330us) server_negotiation.cc:407] Connection header received
0810 14:45:38.144772 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:38.144776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:38.144858 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:38.144942 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:38.145802 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:38.146707 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:38.148112 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:38.148573 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:38.148709 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:38.152746 (+ 4037us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:35145: BlockingRecv error: recv got EOF from 127.24.215.129:35145 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 14:45:38 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:38 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:38.642333 4156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:38.633604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:33987)
0810 14:45:38.634108 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:38.634124 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:38.634240 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:38.634580 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:38.634586 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:38.635128 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:38.635371 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:38.635385 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:38.636665 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:38.636669 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:38.637639 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:38.637652 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:38.638326 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:38.638331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:38.638448 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:38.639029 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:38.639047 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:38.642082 (+ 3035us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":299,"threads_started":1}
W20260810 14:45:38.642594 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:38.633704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:33987 (local address 127.24.215.190:34891)
0810 14:45:38.633972 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:38.633979 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:38.634207 (+ 228us) server_negotiation.cc:407] Connection header received
0810 14:45:38.634345 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:38.634349 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:38.634419 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:38.634508 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:38.635537 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:38.636533 (+ 996us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:38.637761 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:38.638195 (+ 434us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:38.638287 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:38.642306 (+ 4019us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:33987: BlockingRecv error: recv got EOF from 127.24.215.130:33987 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 10 14:45:39 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:39 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:39.118176 4157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:39.109696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:36257)
0810 14:45:39.110191 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:39.110209 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:39.110328 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:39.110620 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:39.110623 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:39.110987 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:39.111206 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:39.111217 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:39.112252 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:39.112257 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:39.113465 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:39.113481 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:39.114229 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:39.114236 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:39.114383 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:39.114976 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:39.114994 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:39.117942 (+ 2948us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":272,"threads_started":1}
W20260810 14:45:39.118417 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:39.109852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:36257 (local address 127.24.215.190:34891)
0810 14:45:39.110117 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:39.110123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:39.110279 (+ 156us) server_negotiation.cc:407] Connection header received
0810 14:45:39.110422 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:39.110425 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:39.110499 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:39.110578 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:39.111337 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:39.112081 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:39.113593 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:39.114014 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:39.114119 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:39.118159 (+ 4040us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:36257: BlockingRecv error: recv got EOF from 127.24.215.131:36257 (error 108)
Metrics: {"server-negotiator.queue_time_us":135}
Aug 10 14:45:39 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:39 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:39.163067 4158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:39.153892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:49595)
0810 14:45:39.154425 (+ 533us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:39.154442 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:39.154577 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:39.155043 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:39.155046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:39.155526 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:39.155719 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:39.155728 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:39.157088 (+ 1360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:39.157096 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:39.158296 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:39.158309 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:39.159007 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:39.159011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:39.159156 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:39.159787 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:39.159804 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:39.162692 (+ 2888us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":297,"threads_started":1}
W20260810 14:45:39.163214 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:39.153949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:49595 (local address 127.24.215.190:34891)
0810 14:45:39.154155 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:39.154159 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:39.154558 (+ 399us) server_negotiation.cc:407] Connection header received
0810 14:45:39.154783 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:39.154788 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:39.154901 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:39.154993 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:39.155865 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:39.156943 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:39.158424 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:39.158881 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:39.158977 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:39.162938 (+ 3961us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:49595: BlockingRecv error: recv got EOF from 127.24.215.129:49595 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 10 14:45:39 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:39 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:39.652804 4159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:39.643324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:38771)
0810 14:45:39.643885 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:39.643900 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:39.644063 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:39.644409 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:39.644415 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:39.644874 (+ 459us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:39.645071 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:39.645081 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:39.646118 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:39.646123 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:39.647041 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:39.647052 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:39.647871 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:39.647877 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:39.648090 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:39.648870 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:39.648898 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:39.652443 (+ 3545us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":300,"threads_started":1}
W20260810 14:45:39.652931 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:39.643411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:38771 (local address 127.24.215.190:34891)
0810 14:45:39.643656 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:39.643666 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:39.643984 (+ 318us) server_negotiation.cc:407] Connection header received
0810 14:45:39.644197 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:39.644202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:39.644277 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:39.644376 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:39.645217 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:39.645973 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:39.647226 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:39.647705 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:39.647837 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:39.652682 (+ 4845us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:38771: BlockingRecv error: recv got EOF from 127.24.215.130:38771 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 10 14:45:40 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:40 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:40.127511 4160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:40.119190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:45129)
0810 14:45:40.119707 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:40.119724 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:40.119834 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:40.120141 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:40.120144 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:40.120554 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:40.120747 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:40.120756 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:40.121757 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:40.121761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:40.122676 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:40.122687 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:40.123402 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:40.123406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:40.123529 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:40.124165 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:40.124183 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:40.127172 (+ 2989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":275,"threads_started":1}
W20260810 14:45:40.127641 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:40.119258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:45129 (local address 127.24.215.190:34891)
0810 14:45:40.119429 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:40.119434 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:40.119796 (+ 362us) server_negotiation.cc:407] Connection header received
0810 14:45:40.119943 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:40.119947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:40.120010 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:40.120091 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:40.120891 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:40.121647 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:40.122798 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:40.123275 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:40.123370 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:40.127388 (+ 4018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:45129: BlockingRecv error: recv got EOF from 127.24.215.131:45129 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 10 14:45:40 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:40 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:40.172940 4161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:40.164046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:53735)
0810 14:45:40.164792 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:40.164808 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:40.164986 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:40.165356 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:40.165361 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:40.165836 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:40.165997 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:40.166006 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:40.167059 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:40.167064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:40.168041 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:40.168053 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:40.168858 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:40.168865 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:40.169074 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:40.169752 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:40.169769 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:40.172589 (+ 2820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":516,"threads_started":1}
W20260810 14:45:40.173058 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:40.164120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:53735 (local address 127.24.215.190:34891)
0810 14:45:40.164342 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:40.164351 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:40.164903 (+ 552us) server_negotiation.cc:407] Connection header received
0810 14:45:40.165125 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:40.165132 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:40.165215 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:40.165320 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:40.166151 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:40.166909 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:40.168190 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:40.168698 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:40.168835 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:40.172806 (+ 3971us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:53735: BlockingRecv error: recv got EOF from 127.24.215.129:53735 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 10 14:45:40 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:40 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:40.662147 4162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:40.653821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:56331)
0810 14:45:40.654388 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:40.654405 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:40.654518 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:40.654789 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:40.654792 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:40.655287 (+ 495us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:40.655455 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:40.655463 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:40.656504 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:40.656508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:40.657464 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:40.657475 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:40.658235 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:40.658241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:40.658432 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:40.659101 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:40.659120 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:40.661929 (+ 2809us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":471,"thread_start_us":353,"threads_started":1}
W20260810 14:45:40.662379 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:40.654063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:56331 (local address 127.24.215.190:34891)
0810 14:45:40.654323 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:40.654329 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:40.654486 (+ 157us) server_negotiation.cc:407] Connection header received
0810 14:45:40.654601 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:40.654604 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:40.654665 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:40.654741 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:40.655582 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:40.656372 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:40.657626 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:40.658074 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:40.658225 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:40.662106 (+ 3881us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:56331: BlockingRecv error: recv got EOF from 127.24.215.130:56331 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 10 14:45:41 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:41 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:41.140955 4163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:41.128597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:38919)
0810 14:45:41.129112 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:41.129129 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:41.129270 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:41.129567 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:41.129571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:41.130047 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:41.130238 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:41.130251 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:41.131901 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:41.131908 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:41.133731 (+ 1823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:41.133751 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:41.135168 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:41.135176 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:41.135412 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:41.136322 (+ 910us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:41.136351 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:41.140597 (+ 4246us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":298,"threads_started":1}
W20260810 14:45:41.141198 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:41.128619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:38919 (local address 127.24.215.190:34891)
0810 14:45:41.128840 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:41.128850 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:41.129212 (+ 362us) server_negotiation.cc:407] Connection header received
0810 14:45:41.129371 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:41.129375 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:41.129437 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:41.129518 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:41.130408 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:41.131681 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:41.133891 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:41.134773 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:41.134913 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:41.140863 (+ 5950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:38919: BlockingRecv error: recv got EOF from 127.24.215.131:38919 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 10 14:45:41 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:41 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:41.183787 4165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:41.174066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:41393)
0810 14:45:41.174734 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:41.174750 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:41.174920 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:41.175330 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:41.175335 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:41.175845 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:41.176111 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:41.176125 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:41.177211 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:41.177217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:41.178316 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:41.178329 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:41.179595 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:41.179600 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:41.179732 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:41.180405 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:41.180421 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:41.183468 (+ 3047us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":352,"threads_started":1}
W20260810 14:45:41.184302 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:41.174097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:41393 (local address 127.24.215.190:34891)
0810 14:45:41.174301 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:41.174310 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:41.174836 (+ 526us) server_negotiation.cc:407] Connection header received
0810 14:45:41.175057 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:41.175063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:41.175190 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:41.175287 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:41.176272 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:41.177064 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:41.178481 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:41.179409 (+ 928us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:41.179570 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:41.183774 (+ 4204us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:41393: BlockingRecv error: recv got EOF from 127.24.215.129:41393 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 10 14:45:41 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:41 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:41.671969 4166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:41.663208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:38855)
0810 14:45:41.663946 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:41.663964 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:41.664104 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:41.664403 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:41.664407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:41.664897 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:41.665082 (+ 185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:41.665092 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:41.666097 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:41.666102 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:41.667148 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:41.667161 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:41.667888 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:41.667893 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:41.668022 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:41.668615 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:41.668644 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:41.671703 (+ 3059us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":421,"threads_started":1}
W20260810 14:45:41.672119 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:41.663349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:38855 (local address 127.24.215.190:34891)
0810 14:45:41.663627 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:41.663634 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:41.664055 (+ 421us) server_negotiation.cc:407] Connection header received
0810 14:45:41.664203 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:41.664207 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:41.664269 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:41.664344 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:41.665215 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:41.665954 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:41.667283 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:41.667704 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:41.667793 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:41.671909 (+ 4116us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:38855: BlockingRecv error: recv got EOF from 127.24.215.130:38855 (error 108)
Metrics: {"server-negotiator.queue_time_us":110}
Aug 10 14:45:42 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:42 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:42.151146 4167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:42.142089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:46195)
0810 14:45:42.142609 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:42.142624 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:42.142739 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:42.143030 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:42.143034 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:42.143512 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:42.143694 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:42.143702 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:42.145040 (+ 1338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:42.145046 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:42.146252 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:42.146266 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:42.146964 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:42.146969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:42.147131 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:42.147756 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:42.147774 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:42.150772 (+ 2998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":275,"threads_started":1}
W20260810 14:45:42.151295 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:42.142131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:46195 (local address 127.24.215.190:34891)
0810 14:45:42.142360 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:42.142409 (+ 49us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:42.142709 (+ 300us) server_negotiation.cc:407] Connection header received
0810 14:45:42.142838 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:42.142842 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:42.142909 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:42.142989 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:42.143855 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:42.144891 (+ 1036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:42.146387 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:42.146804 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:42.146896 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:42.151003 (+ 4107us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:46195: BlockingRecv error: recv got EOF from 127.24.215.131:46195 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 14:45:42 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:42 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:42.193444 4168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:42.184678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:45709)
0810 14:45:42.185214 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:42.185230 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:42.185390 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:42.185770 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:42.185775 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:42.186210 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:42.186391 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:42.186399 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:42.187513 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:42.187518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:42.188508 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:42.188520 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:42.189271 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:42.189276 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:42.189461 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:42.190136 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:42.190154 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:42.193178 (+ 3024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":311,"threads_started":1}
W20260810 14:45:42.193725 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:42.184768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:45709 (local address 127.24.215.190:34891)
0810 14:45:42.185092 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:42.185098 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:42.185319 (+ 221us) server_negotiation.cc:407] Connection header received
0810 14:45:42.185538 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:42.185543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:42.185629 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:42.185735 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:42.186540 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:42.187364 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:42.188662 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:42.189115 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:42.189246 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:42.193349 (+ 4103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:45709: BlockingRecv error: recv got EOF from 127.24.215.129:45709 (error 108)
Metrics: {"server-negotiator.queue_time_us":159}
Aug 10 14:45:42 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:42 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:42.682045 4169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:42.673042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:58121)
0810 14:45:42.673513 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:42.673530 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:42.673643 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:42.673945 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:42.673948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:42.674330 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:42.674518 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:42.674527 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:42.675633 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:42.675638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:42.676596 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:42.676607 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:42.677326 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:42.677331 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:42.677463 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:42.678164 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:42.678182 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:42.681676 (+ 3494us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":255,"threads_started":1}
W20260810 14:45:42.682081 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:42.673091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:58121 (local address 127.24.215.190:34891)
0810 14:45:42.673331 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:42.673339 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:42.673611 (+ 272us) server_negotiation.cc:407] Connection header received
0810 14:45:42.673745 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:42.673749 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:42.673819 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:42.673910 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:42.674678 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:42.675492 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:42.676741 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:42.677180 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:42.677285 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:42.681897 (+ 4612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:58121: BlockingRecv error: recv got EOF from 127.24.215.130:58121 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 14:45:43 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:43 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.131@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:43.165966 4171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:43.152118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.131:34641)
0810 14:45:43.152602 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:43.152618 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:43.152735 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:43.153021 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:43.153024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:43.153409 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:43.153588 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:43.153597 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:43.154549 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:43.154553 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:43.155610 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:43.155623 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:43.156302 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:43.156306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:43.156446 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:43.162082 (+ 5636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:43.162108 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:43.165614 (+ 3506us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":283,"threads_started":1}
W20260810 14:45:43.166136 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:43.152213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:34641 (local address 127.24.215.190:34891)
0810 14:45:43.152480 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:43.152486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:43.152698 (+ 212us) server_negotiation.cc:407] Connection header received
0810 14:45:43.152833 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:43.152836 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:43.152899 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:43.152985 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:43.153706 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:43.154422 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:43.155734 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:43.156177 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:43.156269 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:43.165876 (+ 9607us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.131:34641: BlockingRecv error: recv got EOF from 127.24.215.131:34641 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 10 14:45:43 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:43 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.129@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:43.203404 4172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:43.194637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.129:52013)
0810 14:45:43.195186 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:43.195203 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:43.195331 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:43.195686 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:43.195691 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:43.196096 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:43.196269 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:43.196278 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:43.197420 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:43.197425 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:43.198392 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:43.198404 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:43.199407 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:43.199412 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:43.199549 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:43.200189 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:43.200207 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:43.203015 (+ 2808us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":276,"threads_started":1}
W20260810 14:45:43.203529 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:43.194790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:52013 (local address 127.24.215.190:34891)
0810 14:45:43.195030 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:43.195036 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:43.195288 (+ 252us) server_negotiation.cc:407] Connection header received
0810 14:45:43.195466 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:43.195470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:43.195555 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:43.195636 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:43.196389 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:43.197258 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:43.198559 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:43.199057 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:43.199157 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:43.203263 (+ 4106us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.129:52013: BlockingRecv error: recv got EOF from 127.24.215.129:52013 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 10 14:45:43 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
Aug 10 14:45:43 dist-test-slave-vkjp krb5kdc[2902](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.24.215.130@KRBTEST.COM for oryx/127.24.215.190@KRBTEST.COM, Server not found in Kerberos database
I20260810 14:45:43.691919 4173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:43.683190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34891 (local address 127.24.215.130:56683)
0810 14:45:43.683729 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:43.683747 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:43.683876 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:43.684198 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:43.684202 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:43.684705 (+ 503us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:43.684884 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:43.684892 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:43.685928 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:43.685932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:43.686894 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:43.686906 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:43.687643 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:43.687648 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:43.687762 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:43.688370 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:43.688388 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:43.691537 (+ 3149us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.24.215.190:34891: Server oryx/127.24.215.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":433,"thread_start_us":317,"threads_started":1}
W20260810 14:45:43.692023 3799 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:43.683360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:56683 (local address 127.24.215.190:34891)
0810 14:45:43.683594 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:43.683600 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:43.683837 (+ 237us) server_negotiation.cc:407] Connection header received
0810 14:45:43.683983 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:43.683987 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:43.684048 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:43.684128 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:43.685012 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:43.685760 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:43.687027 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:43.687511 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:43.687624 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:43.691765 (+ 4141us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.24.215.130:56683: BlockingRecv error: recv got EOF from 127.24.215.130:56683 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 14:45:43.980705 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 3610
I20260810 14:45:43.993857 4175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:43.981834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37996 (local address 127.24.215.129:43687)
0810 14:45:43.982448 (+ 614us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:43.982456 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:43.982485 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:45:43.982549 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:43.982556 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:43.982787 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:43.982949 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:43.984737 (+ 1788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:43.985789 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:43.986969 (+ 1180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:43.987569 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:43.987664 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:43.990123 (+ 2459us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:43.990131 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:43.990145 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:43.990177 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:43.993109 (+ 2932us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.215.129@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 14:45:43.993511 (+ 402us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37996: Request ticket server kudu/127.24.215.129@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":443,"thread_start_us":319,"threads_started":1}
W20260810 14:45:43.993983 4175 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37996: Request ticket server kudu/127.24.215.129@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 14:45:43.993927 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 3610: Not authorized: Client connection negotiation failed: client connection to 127.24.215.129:43687: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.215.129@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 14:45:43.994134 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 3610
I20260810 14:45:44.018649 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 3802
I20260810 14:45:44.031571 4176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:44.019868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51520 (local address 127.24.215.130:43047)
0810 14:45:44.020494 (+ 626us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:44.020502 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:44.020534 (+ 32us) server_negotiation.cc:407] Connection header received
0810 14:45:44.020599 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:44.020606 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:44.020833 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:44.021055 (+ 222us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:44.022415 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:44.023583 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:44.024805 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:44.025424 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:44.025578 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:44.028271 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:44.028280 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:44.028293 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:44.028325 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:44.030799 (+ 2474us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.215.130@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 14:45:44.031196 (+ 397us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51520: Request ticket server kudu/127.24.215.130@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":309,"threads_started":1}
W20260810 14:45:44.031577 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 3802: Not authorized: Client connection negotiation failed: client connection to 127.24.215.130:43047: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.215.130@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 14:45:44.031684 4176 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51520: Request ticket server kudu/127.24.215.130@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 14:45:44.031755 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 3802
I20260810 14:45:44.052100 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 3960
I20260810 14:45:44.064296 4178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:44.053153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51428 (local address 127.24.215.131:44287)
0810 14:45:44.053784 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:44.053791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:44.053819 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:45:44.053890 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:44.053897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:44.054117 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:44.054284 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:44.055676 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:44.056734 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:44.057950 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:44.058500 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:44.058595 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:44.061083 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:44.061091 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:44.061106 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:44.061136 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:44.063792 (+ 2656us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.215.131@KRBTEST.COM not found in keytab (ticket kvno 2)
0810 14:45:44.064051 (+ 259us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:51428: Request ticket server kudu/127.24.215.131@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":342,"threads_started":1}
W20260810 14:45:44.064417 4178 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:51428: Request ticket server kudu/127.24.215.131@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260810 14:45:44.064471 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 3960: Not authorized: Client connection negotiation failed: client connection to 127.24.215.131:44287: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.24.215.131@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260810 14:45:44.064623 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 3960
I20260810 14:45:44.087400 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 3529
I20260810 14:45:44.088814 3602 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:44.209517 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 3529
2026-08-10T14:45:44Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (21381 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-10T14:45:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:45:44Z Disabled control of system clock
I20260810 14:45:44.300823 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:38615
--webserver_interface=127.24.215.190
--webserver_port=0
--builtin_ntp_servers=127.24.215.148:33175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:38615
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260810 14:45:44.644310 4184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:44.644609 4184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:44.644758 4184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:44.652281 4184 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:45:44.652387 4184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:44.652459 4184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:44.652551 4184 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:45:44.652614 4184 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:45:44.662179 4184 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:33175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:38615
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:38615
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:44.663619 4184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:44.665437 4184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:44.677410 4191 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:44.677405 4192 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:44.678136 4195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:44.678412 4184 server_base.cc:1034] running on GCE node
I20260810 14:45:44.679350 4184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:44.680819 4184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:44.682104 4184 hybrid_clock.cc:648] HybridClock initialized: now 1786373144682043 us; error 47 us; skew 500 ppm
I20260810 14:45:44.682566 4184 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:45:44.685141 4184 webserver.cc:460] Webserver started at http://127.24.215.190:40373/ using document root <none> and password file <none>
I20260810 14:45:44.685920 4184 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:44.686069 4184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:44.686453 4184 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:44.689693 4184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "ec3274da1cd64a4bab7e1592d30861fd"
format_stamp: "Formatted at 2026-08-10 14:45:44 on dist-test-slave-vkjp"
server_key: "0b1e4e4b0826e4e20fae9c6907e00e24"
server_key_iv: "bd3feca7e27c218e91d8e39e277d2067"
server_key_version: "encryptionkey@0"
I20260810 14:45:44.690578 4184 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "ec3274da1cd64a4bab7e1592d30861fd"
format_stamp: "Formatted at 2026-08-10 14:45:44 on dist-test-slave-vkjp"
server_key: "0b1e4e4b0826e4e20fae9c6907e00e24"
server_key_iv: "bd3feca7e27c218e91d8e39e277d2067"
server_key_version: "encryptionkey@0"
I20260810 14:45:44.696251 4184 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 14:45:44.700074 4201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:44.701539 4184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:45:44.701743 4184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "ec3274da1cd64a4bab7e1592d30861fd"
format_stamp: "Formatted at 2026-08-10 14:45:44 on dist-test-slave-vkjp"
server_key: "0b1e4e4b0826e4e20fae9c6907e00e24"
server_key_iv: "bd3feca7e27c218e91d8e39e277d2067"
server_key_version: "encryptionkey@0"
I20260810 14:45:44.701985 4184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:44.722297 4184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:44.723254 4184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:44.723574 4184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:44.742559 4184 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:38615
I20260810 14:45:44.742583 4262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:38615 every 8 connection(s)
I20260810 14:45:44.744043 4184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:45:44.747812 4263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:44.753981 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 4184
I20260810 14:45:44.754201 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:45:44.754846 25438 external_mini_cluster.cc:1428] Setting key 21346461220ccec82584b6432dca240e
I20260810 14:45:44.758963 4263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd: Bootstrap starting.
I20260810 14:45:44.762051 4263 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:44.763118 4263 log.cc:824] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:44.766470 4263 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd: No bootstrap required, opened a new log
I20260810 14:45:44.769631 4266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:44.756969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42790 (local address 127.24.215.190:38615)
0810 14:45:44.757945 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:44.757958 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:44.757999 (+ 41us) server_negotiation.cc:407] Connection header received
0810 14:45:44.759752 (+ 1753us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:44.759776 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:44.760208 (+ 432us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:44.760607 (+ 399us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:44.761292 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:44.762716 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:44.764494 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:44.765507 (+ 1013us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:44.765708 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:44.766828 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:44.766838 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:44.766856 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:45:44.767615 (+ 759us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:44.767717 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:45:44.767744 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:44.767877 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:44.768049 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 14:45:44.768366 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":335,"threads_started":1}
I20260810 14:45:44.774901 4263 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3274da1cd64a4bab7e1592d30861fd" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 38615 } }
I20260810 14:45:44.775285 4263 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:44.775360 4263 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec3274da1cd64a4bab7e1592d30861fd, State: Initialized, Role: FOLLOWER
I20260810 14:45:44.775832 4263 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3274da1cd64a4bab7e1592d30861fd" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 38615 } }
I20260810 14:45:44.776019 4263 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:45:44.776119 4263 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:45:44.776245 4263 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:44.778406 4263 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3274da1cd64a4bab7e1592d30861fd" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 38615 } }
I20260810 14:45:44.778853 4263 leader_election.cc:302] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec3274da1cd64a4bab7e1592d30861fd; no voters:
I20260810 14:45:44.779487 4263 leader_election.cc:288] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:45:44.779632 4268 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:45:44.780403 4268 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [term 1 LEADER]: Becoming Leader. State: Replica: ec3274da1cd64a4bab7e1592d30861fd, State: Running, Role: LEADER
I20260810 14:45:44.780951 4268 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3274da1cd64a4bab7e1592d30861fd" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 38615 } }
I20260810 14:45:44.781570 4263 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:45:44.784025 4269 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec3274da1cd64a4bab7e1592d30861fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3274da1cd64a4bab7e1592d30861fd" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 38615 } } }
I20260810 14:45:44.784394 4269 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:44.784819 4271 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec3274da1cd64a4bab7e1592d30861fd. Latest consensus state: current_term: 1 leader_uuid: "ec3274da1cd64a4bab7e1592d30861fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec3274da1cd64a4bab7e1592d30861fd" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 38615 } } }
I20260810 14:45:44.785166 4271 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:44.786942 4277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:45:44.791680 4277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:45:44.800936 4277 catalog_manager.cc:1347] Generated new cluster ID: 34a898a5e18340b18ce5673a8e2f99f1
I20260810 14:45:44.801088 4277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:45:44.819308 4277 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:45:44.820565 4277 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:45:44.828301 4277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec3274da1cd64a4bab7e1592d30861fd: Generated new TSK 0
I20260810 14:45:44.829370 4277 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:45:44.843346 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:38615
--builtin_ntp_servers=127.24.215.148:33175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260810 14:45:45.186326 4294 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:45.186681 4294 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:45.186836 4294 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:45.194242 4294 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:45.194344 4294 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:45.194535 4294 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:45:45.203761 4294 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:33175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:38615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:45.205088 4294 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:45.207146 4294 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:45.220307 4299 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:45.220427 4302 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:45.220458 4300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:45.221329 4294 server_base.cc:1034] running on GCE node
I20260810 14:45:45.221951 4294 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:45.222922 4294 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:45.224128 4294 hybrid_clock.cc:648] HybridClock initialized: now 1786373145224092 us; error 51 us; skew 500 ppm
I20260810 14:45:45.224503 4294 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:45:45.226917 4294 webserver.cc:460] Webserver started at http://127.24.215.129:36451/ using document root <none> and password file <none>
I20260810 14:45:45.227682 4294 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:45.227775 4294 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:45.228049 4294 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:45.230919 4294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "92afc73efde94f61804a3bff2bf945cc"
format_stamp: "Formatted at 2026-08-10 14:45:45 on dist-test-slave-vkjp"
server_key: "2b9a86b0f47f3769abf427ed921211e3"
server_key_iv: "547afb34c0063e2fa75e153f44273451"
server_key_version: "encryptionkey@0"
I20260810 14:45:45.231751 4294 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "92afc73efde94f61804a3bff2bf945cc"
format_stamp: "Formatted at 2026-08-10 14:45:45 on dist-test-slave-vkjp"
server_key: "2b9a86b0f47f3769abf427ed921211e3"
server_key_iv: "547afb34c0063e2fa75e153f44273451"
server_key_version: "encryptionkey@0"
I20260810 14:45:45.236831 4294 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:45:45.240264 4311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:45.241639 4294 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:45:45.241827 4294 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "92afc73efde94f61804a3bff2bf945cc"
format_stamp: "Formatted at 2026-08-10 14:45:45 on dist-test-slave-vkjp"
server_key: "2b9a86b0f47f3769abf427ed921211e3"
server_key_iv: "547afb34c0063e2fa75e153f44273451"
server_key_version: "encryptionkey@0"
I20260810 14:45:45.242058 4294 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:45.255002 4294 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:45.255925 4294 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:45.256246 4294 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:45.257207 4294 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:45.258711 4294 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:45.258836 4294 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:45.258962 4294 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:45.259052 4294 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:45.294227 4294 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:38907
I20260810 14:45:45.294317 4441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:38907 every 8 connection(s)
I20260810 14:45:45.295632 4294 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:45:45.302035 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 4294
I20260810 14:45:45.302268 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:45:45.303148 25438 external_mini_cluster.cc:1428] Setting key 01b0ac9ade551d4381de0dc7b8383bc9
I20260810 14:45:45.306496 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:38615
--builtin_ntp_servers=127.24.215.148:33175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 14:45:45.310065 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:45.298245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:33879 (local address 127.24.215.190:38615)
0810 14:45:45.298734 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:45.298739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:45.299804 (+ 1065us) server_negotiation.cc:407] Connection header received
0810 14:45:45.301634 (+ 1830us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:45.301639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:45.301709 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:45.301807 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:45.303346 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:45.304222 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:45.306778 (+ 2556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:45.307548 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:45.307696 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:45.309112 (+ 1416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:45.309120 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:45.309128 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:45:45.309153 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:45.309207 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:45:45.309225 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:45.309315 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:45.309538 (+ 223us) server_negotiation.cc:299] Negotiation successful
0810 14:45:45.309651 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":267,"threads_started":1}
I20260810 14:45:45.310925 4445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:45.298684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:38615 (local address 127.24.215.129:33879)
0810 14:45:45.299649 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:45.299685 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:45.301337 (+ 1652us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:45.302101 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:45.302112 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:45.302239 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:45.303152 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:45.303179 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:45.304440 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:45.304445 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:45.306573 (+ 2128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:45.306589 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:45.307788 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:45.307793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:45.308028 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:45.308764 (+ 736us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:45:45.308767 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:45.308818 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:45:45.308911 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:45.309328 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:45.309336 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:45.309342 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:45:45.309492 (+ 150us) client_negotiation.cc:240] Negotiation successful
0810 14:45:45.309782 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":324,"threads_started":1}
I20260810 14:45:45.312588 4442 heartbeater.cc:347] Connected to a master server at 127.24.215.190:38615
I20260810 14:45:45.312908 4442 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:45.313659 4442 heartbeater.cc:511] Master 127.24.215.190:38615 requested a full tablet report, sending...
I20260810 14:45:45.315778 4221 ts_manager.cc:194] Registered new tserver with Master: 92afc73efde94f61804a3bff2bf945cc (127.24.215.129:38907)
I20260810 14:45:45.317296 4221 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.215.129:33879
W20260810 14:45:45.652649 4446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:45.652923 4446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:45.653059 4446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:45.660261 4446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:45.660364 4446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:45.660522 4446 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:45:45.669708 4446 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:33175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:38615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:45.671010 4446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:45.672904 4446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:45.685926 4453 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:45.685963 4454 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:45.685985 4456 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:45.686764 4446 server_base.cc:1034] running on GCE node
I20260810 14:45:45.687438 4446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:45.688380 4446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:45.689651 4446 hybrid_clock.cc:648] HybridClock initialized: now 1786373145689574 us; error 44 us; skew 500 ppm
I20260810 14:45:45.689977 4446 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:45:45.692328 4446 webserver.cc:460] Webserver started at http://127.24.215.130:41069/ using document root <none> and password file <none>
I20260810 14:45:45.693037 4446 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:45.693171 4446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:45.693519 4446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:45.696517 4446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "8d50847a359f4206806311901f067037"
format_stamp: "Formatted at 2026-08-10 14:45:45 on dist-test-slave-vkjp"
server_key: "f661b5297cb56877ce2dd66fd9994116"
server_key_iv: "7b97e114f15969a47c3d3d6a5d774012"
server_key_version: "encryptionkey@0"
I20260810 14:45:45.697350 4446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "8d50847a359f4206806311901f067037"
format_stamp: "Formatted at 2026-08-10 14:45:45 on dist-test-slave-vkjp"
server_key: "f661b5297cb56877ce2dd66fd9994116"
server_key_iv: "7b97e114f15969a47c3d3d6a5d774012"
server_key_version: "encryptionkey@0"
I20260810 14:45:45.702605 4446 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 14:45:45.706496 4464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:45.708086 4446 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:45:45.708297 4446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "8d50847a359f4206806311901f067037"
format_stamp: "Formatted at 2026-08-10 14:45:45 on dist-test-slave-vkjp"
server_key: "f661b5297cb56877ce2dd66fd9994116"
server_key_iv: "7b97e114f15969a47c3d3d6a5d774012"
server_key_version: "encryptionkey@0"
I20260810 14:45:45.708560 4446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:45.729938 4446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:45.730895 4446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:45.731341 4446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:45.732438 4446 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:45.734110 4446 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:45.734242 4446 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:45.734398 4446 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:45.734493 4446 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:45.772912 4446 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:42079
I20260810 14:45:45.773018 4595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:42079 every 8 connection(s)
I20260810 14:45:45.774395 4446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:45:45.784549 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 4446
I20260810 14:45:45.784756 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:45:45.785315 25438 external_mini_cluster.cc:1428] Setting key dc4b9f03569f425de407fc45f3b36b3c
I20260810 14:45:45.787765 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:45.777387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:51799 (local address 127.24.215.190:38615)
0810 14:45:45.777607 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:45.777612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:45.779142 (+ 1530us) server_negotiation.cc:407] Connection header received
0810 14:45:45.780881 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:45.780886 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:45.780955 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:45.781073 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:45.782180 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:45.782931 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:45.784574 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:45.785224 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:45.785375 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:45.786808 (+ 1433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:45.786813 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:45.786817 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:45:45.786835 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:45.786865 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:45:45.786878 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:45.786977 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:45.787337 (+ 360us) server_negotiation.cc:299] Negotiation successful
0810 14:45:45.787462 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260810 14:45:45.788463 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:38615
--builtin_ntp_servers=127.24.215.148:33175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260810 14:45:45.788884 4598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:45.777858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:38615 (local address 127.24.215.130:51799)
0810 14:45:45.778925 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:45.778972 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:45.780610 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:45.781267 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:45.781275 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:45.781387 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:45.781951 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:45.781968 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:45.783072 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:45.783076 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:45.784416 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:45.784429 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:45.785500 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:45.785506 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:45.785710 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:45.786456 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:45:45.786459 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:45.786501 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:45:45.786585 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:45.786984 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:45.786997 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:45.787007 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:45:45.787300 (+ 293us) client_negotiation.cc:240] Negotiation successful
0810 14:45:45.787665 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":293,"threads_started":1}
I20260810 14:45:45.790403 4596 heartbeater.cc:347] Connected to a master server at 127.24.215.190:38615
I20260810 14:45:45.790710 4596 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:45.791569 4596 heartbeater.cc:511] Master 127.24.215.190:38615 requested a full tablet report, sending...
I20260810 14:45:45.793038 4221 ts_manager.cc:194] Registered new tserver with Master: 8d50847a359f4206806311901f067037 (127.24.215.130:42079)
I20260810 14:45:45.794236 4221 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.215.130:51799
W20260810 14:45:46.129894 4601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:46.130167 4601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:46.130332 4601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:46.137583 4601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:46.137692 4601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:46.137878 4601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:45:46.147127 4601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:33175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:38615
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:46.148454 4601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:46.150251 4601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:46.163687 4609 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:46.163754 4608 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:46.163687 4611 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:46.169847 4601 server_base.cc:1034] running on GCE node
I20260810 14:45:46.170495 4601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:46.171504 4601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:46.172752 4601 hybrid_clock.cc:648] HybridClock initialized: now 1786373146172702 us; error 57 us; skew 500 ppm
I20260810 14:45:46.173076 4601 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:45:46.175446 4601 webserver.cc:460] Webserver started at http://127.24.215.131:45315/ using document root <none> and password file <none>
I20260810 14:45:46.176163 4601 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:46.176293 4601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:46.176636 4601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:46.179489 4601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "427d30c5a1804d32aa5c36d80bfd615b"
format_stamp: "Formatted at 2026-08-10 14:45:46 on dist-test-slave-vkjp"
server_key: "19ccc59658150d9d4b3808fb54c19229"
server_key_iv: "a6a0991719426aff60317a78bf6a6c57"
server_key_version: "encryptionkey@0"
I20260810 14:45:46.180287 4601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "427d30c5a1804d32aa5c36d80bfd615b"
format_stamp: "Formatted at 2026-08-10 14:45:46 on dist-test-slave-vkjp"
server_key: "19ccc59658150d9d4b3808fb54c19229"
server_key_iv: "a6a0991719426aff60317a78bf6a6c57"
server_key_version: "encryptionkey@0"
I20260810 14:45:46.185307 4601 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260810 14:45:46.188741 4617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:46.190008 4601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 14:45:46.190186 4601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "427d30c5a1804d32aa5c36d80bfd615b"
format_stamp: "Formatted at 2026-08-10 14:45:46 on dist-test-slave-vkjp"
server_key: "19ccc59658150d9d4b3808fb54c19229"
server_key_iv: "a6a0991719426aff60317a78bf6a6c57"
server_key_version: "encryptionkey@0"
I20260810 14:45:46.190397 4601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:46.214272 4601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:46.215238 4601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:46.215576 4601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:46.216591 4601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:46.218022 4601 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:46.218152 4601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:46.218286 4601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:46.218375 4601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:46.252559 4601 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:37257
I20260810 14:45:46.252650 4751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:37257 every 8 connection(s)
I20260810 14:45:46.253840 4601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:45:46.256203 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 4601
I20260810 14:45:46.256410 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:45:46.256986 25438 external_mini_cluster.cc:1428] Setting key 33e6efbc723f27b7611222d17eebb803
I20260810 14:45:46.266299 4444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:46.256685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:33555 (local address 127.24.215.190:38615)
0810 14:45:46.256872 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:46.256876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:46.258598 (+ 1722us) server_negotiation.cc:407] Connection header received
0810 14:45:46.260405 (+ 1807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:46.260409 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:46.260475 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:46.260559 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:46.261567 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:46.262309 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:46.263904 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:46.264293 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:46.264390 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:46.265659 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:46.265663 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:46.265666 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:45:46.265684 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:46.265713 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:45:46.265723 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:46.265790 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:46.265980 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 14:45:46.266055 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 14:45:46.267344 4754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:46.257302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:38615 (local address 127.24.215.131:33555)
0810 14:45:46.258424 (+ 1122us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:46.258476 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:46.260167 (+ 1691us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:46.260748 (+ 581us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:46.260756 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:46.260842 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:46.261420 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:46.261436 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:46.262444 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:46.262447 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:46.263772 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:46.263784 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:46.264472 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:46.264476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:46.264674 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:46.265368 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:45:46.265370 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:46.265407 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:45:46.265489 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:46.265805 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:46.265812 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:46.265818 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:45:46.265954 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 14:45:46.266237 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":606,"thread_start_us":392,"threads_started":1}
I20260810 14:45:46.268709 4752 heartbeater.cc:347] Connected to a master server at 127.24.215.190:38615
I20260810 14:45:46.269030 4752 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:46.269800 4752 heartbeater.cc:511] Master 127.24.215.190:38615 requested a full tablet report, sending...
I20260810 14:45:46.271353 4220 ts_manager.cc:194] Registered new tserver with Master: 427d30c5a1804d32aa5c36d80bfd615b (127.24.215.131:37257)
I20260810 14:45:46.272431 4220 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.215.131:33555
I20260810 14:45:46.272466 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 14:45:46.279594 4444 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:46.277878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42792 (local address 127.24.215.190:38615)
0810 14:45:46.278042 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:46.278046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:46.278593 (+ 547us) server_negotiation.cc:407] Connection header received
0810 14:45:46.278735 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:46.278738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:46.278813 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:46.278895 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:46.279374 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42792: BlockingRecv error: recv got EOF from 127.0.0.1:42792 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 14:45:46.280001 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 4294
I20260810 14:45:46.287984 4762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:46.281015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34480 (local address 127.24.215.129:38907)
0810 14:45:46.281575 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:46.281582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:46.281609 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:45:46.281671 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:46.281677 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:46.281869 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:46.281997 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:46.282322 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:46.283431 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:46.284697 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:46.285243 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:46.285364 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:46.286407 (+ 1043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:46.286420 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:46.286439 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:45:46.287046 (+ 607us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:46.287149 (+ 103us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:45:46.287178 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:46.287264 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:46.287402 (+ 138us) server_negotiation.cc:299] Negotiation successful
0810 14:45:46.287632 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":245,"threads_started":1}
I20260810 14:45:46.289253 4437 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:46.369509 4442 heartbeater.cc:503] Master 127.24.215.190:38615 was elected leader, sending a full tablet report...
I20260810 14:45:46.375327 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 4294
I20260810 14:45:46.396538 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 4446
I20260810 14:45:46.404182 4764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:46.397481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34794 (local address 127.24.215.130:42079)
0810 14:45:46.398011 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:46.398018 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:46.398045 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:45:46.398116 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:46.398124 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:46.398347 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:46.398494 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:46.398870 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:46.399907 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:46.401163 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:46.401709 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:46.401810 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:46.402751 (+ 941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:46.402760 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:46.402774 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:45:46.403428 (+ 654us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:46.403488 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:45:46.403509 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:46.403597 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:46.403767 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 14:45:46.403890 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":382,"thread_start_us":287,"threads_started":1}
I20260810 14:45:46.405483 4589 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:46.486572 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 4446
I20260810 14:45:46.506793 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 4601
I20260810 14:45:46.515317 4767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:46.507808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51522 (local address 127.24.215.131:37257)
0810 14:45:46.508466 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:46.508473 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:46.508504 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:45:46.508565 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:46.508571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:46.508787 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:46.508995 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:46.509495 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:46.510558 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:46.511929 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:46.512470 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:46.512623 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:46.513782 (+ 1159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:46.513791 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:46.513805 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:45:46.514382 (+ 577us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:46.514433 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:45:46.514452 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:46.514579 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:46.514762 (+ 183us) server_negotiation.cc:299] Negotiation successful
0810 14:45:46.514901 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":351,"threads_started":1}
I20260810 14:45:46.516584 4746 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:46.601984 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 4601
I20260810 14:45:46.624166 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 4184
I20260810 14:45:46.625521 4258 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:46.730211 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 4184
2026-08-10T14:45:46Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2512 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-10T14:45:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:45:46Z Disabled control of system clock
I20260810 14:45:46.813643 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:41737
--webserver_interface=127.24.215.190
--webserver_port=0
--builtin_ntp_servers=127.24.215.148:45939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:41737
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 14:45:47.150075 4773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:47.150379 4773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:47.150523 4773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:47.157727 4773 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:45:47.157824 4773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:47.157920 4773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:47.158034 4773 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:45:47.158116 4773 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:45:47.167318 4773 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:41737
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:41737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:47.168601 4773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:47.170416 4773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:47.182511 4780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:47.182533 4779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:47.182471 4782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:47.183624 4773 server_base.cc:1034] running on GCE node
I20260810 14:45:47.184388 4773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:47.185729 4773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:47.186998 4773 hybrid_clock.cc:648] HybridClock initialized: now 1786373147186947 us; error 47 us; skew 500 ppm
I20260810 14:45:47.187427 4773 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:45:47.189779 4773 webserver.cc:460] Webserver started at http://127.24.215.190:36093/ using document root <none> and password file <none>
I20260810 14:45:47.190464 4773 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:47.190595 4773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:47.190937 4773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:47.193787 4773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "ad0445105059443fa914bcd81f6828bf"
format_stamp: "Formatted at 2026-08-10 14:45:47 on dist-test-slave-vkjp"
server_key: "fda1391576b905c157313f68af2012fa"
server_key_iv: "b90017caf22df3b8caa53c93779b26c9"
server_key_version: "encryptionkey@0"
I20260810 14:45:47.194610 4773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "ad0445105059443fa914bcd81f6828bf"
format_stamp: "Formatted at 2026-08-10 14:45:47 on dist-test-slave-vkjp"
server_key: "fda1391576b905c157313f68af2012fa"
server_key_iv: "b90017caf22df3b8caa53c93779b26c9"
server_key_version: "encryptionkey@0"
I20260810 14:45:47.199748 4773 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 14:45:47.203186 4789 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:47.204455 4773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 14:45:47.204699 4773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "ad0445105059443fa914bcd81f6828bf"
format_stamp: "Formatted at 2026-08-10 14:45:47 on dist-test-slave-vkjp"
server_key: "fda1391576b905c157313f68af2012fa"
server_key_iv: "b90017caf22df3b8caa53c93779b26c9"
server_key_version: "encryptionkey@0"
I20260810 14:45:47.204933 4773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:47.210630 4773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:47.211512 4773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:47.211822 4773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:47.230091 4773 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:41737
I20260810 14:45:47.230123 4852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:41737 every 8 connection(s)
I20260810 14:45:47.231465 4773 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:45:47.235301 4853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:47.237432 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 4773
I20260810 14:45:47.237627 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:45:47.238452 25438 external_mini_cluster.cc:1428] Setting key d78b133f5c932feb7d1b1542850a38d0
I20260810 14:45:47.247464 4858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:47.240700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43178 (local address 127.24.215.190:41737)
0810 14:45:47.241778 (+ 1078us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:47.241790 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:47.241834 (+ 44us) server_negotiation.cc:407] Connection header received
0810 14:45:47.243749 (+ 1915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:47.243773 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:47.244196 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:47.244582 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:47.245708 (+ 1126us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:47.245714 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:47.245729 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:45:47.245743 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:47.245805 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:45:47.245819 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:47.246100 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:47.246276 (+ 176us) server_negotiation.cc:299] Negotiation successful
0810 14:45:47.246509 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":354,"threads_started":1}
I20260810 14:45:47.248313 4853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf: Bootstrap starting.
I20260810 14:45:47.252074 4853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:47.253064 4853 log.cc:824] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:47.255959 4853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf: No bootstrap required, opened a new log
I20260810 14:45:47.263948 4853 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0445105059443fa914bcd81f6828bf" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41737 } }
I20260810 14:45:47.264286 4853 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:47.264365 4853 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad0445105059443fa914bcd81f6828bf, State: Initialized, Role: FOLLOWER
I20260810 14:45:47.264866 4853 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0445105059443fa914bcd81f6828bf" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41737 } }
I20260810 14:45:47.265059 4853 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:45:47.265156 4853 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:45:47.265287 4853 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:47.267506 4853 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0445105059443fa914bcd81f6828bf" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41737 } }
I20260810 14:45:47.268129 4853 leader_election.cc:302] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad0445105059443fa914bcd81f6828bf; no voters:
I20260810 14:45:47.268826 4853 leader_election.cc:288] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:45:47.268930 4860 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:45:47.269726 4860 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [term 1 LEADER]: Becoming Leader. State: Replica: ad0445105059443fa914bcd81f6828bf, State: Running, Role: LEADER
I20260810 14:45:47.270236 4860 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0445105059443fa914bcd81f6828bf" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41737 } }
I20260810 14:45:47.271018 4853 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:45:47.273459 4861 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad0445105059443fa914bcd81f6828bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0445105059443fa914bcd81f6828bf" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41737 } } }
I20260810 14:45:47.273450 4863 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad0445105059443fa914bcd81f6828bf. Latest consensus state: current_term: 1 leader_uuid: "ad0445105059443fa914bcd81f6828bf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad0445105059443fa914bcd81f6828bf" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 41737 } } }
I20260810 14:45:47.273967 4861 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:47.273972 4863 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:47.274794 4867 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:45:47.280248 4867 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:45:47.291460 4867 catalog_manager.cc:1347] Generated new cluster ID: 7f9d75e1229b4411aed7a81c0440b30f
I20260810 14:45:47.291595 4867 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:45:47.306986 4867 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:45:47.307447 4867 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:45:47.313797 4867 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad0445105059443fa914bcd81f6828bf: Generated new TSK 0
I20260810 14:45:47.314546 4867 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:45:47.319360 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:41737
--builtin_ntp_servers=127.24.215.148:45939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260810 14:45:47.684259 4882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:47.684540 4882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:47.684716 4882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:47.692097 4882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:47.692212 4882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:47.692421 4882 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:45:47.701640 4882 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:41737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:47.703059 4882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:47.704813 4882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:47.718153 4887 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:47.718123 4888 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:47.718191 4890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:47.718935 4882 server_base.cc:1034] running on GCE node
I20260810 14:45:47.719620 4882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:47.720547 4882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:47.721776 4882 hybrid_clock.cc:648] HybridClock initialized: now 1786373147721725 us; error 58 us; skew 500 ppm
I20260810 14:45:47.722095 4882 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:45:47.724360 4882 webserver.cc:460] Webserver started at http://127.24.215.129:37307/ using document root <none> and password file <none>
I20260810 14:45:47.725056 4882 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:47.725205 4882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:47.725530 4882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:47.728456 4882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "b45f76bc69c2402dbb56122166cb0e79"
format_stamp: "Formatted at 2026-08-10 14:45:47 on dist-test-slave-vkjp"
server_key: "1bc19e44e57daf56fc273705ca8a6d4d"
server_key_iv: "4e62aca32c4557a730e5be8d728fb94b"
server_key_version: "encryptionkey@0"
I20260810 14:45:47.729270 4882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "b45f76bc69c2402dbb56122166cb0e79"
format_stamp: "Formatted at 2026-08-10 14:45:47 on dist-test-slave-vkjp"
server_key: "1bc19e44e57daf56fc273705ca8a6d4d"
server_key_iv: "4e62aca32c4557a730e5be8d728fb94b"
server_key_version: "encryptionkey@0"
I20260810 14:45:47.734280 4882 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:45:47.737772 4896 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:47.739120 4882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:45:47.739301 4882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "b45f76bc69c2402dbb56122166cb0e79"
format_stamp: "Formatted at 2026-08-10 14:45:47 on dist-test-slave-vkjp"
server_key: "1bc19e44e57daf56fc273705ca8a6d4d"
server_key_iv: "4e62aca32c4557a730e5be8d728fb94b"
server_key_version: "encryptionkey@0"
I20260810 14:45:47.739534 4882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:47.745287 4882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:47.746073 4882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:47.746372 4882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:47.747318 4882 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:47.748724 4882 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:47.748849 4882 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:47.748978 4882 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:47.749066 4882 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:47.782694 4882 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:35143
I20260810 14:45:47.782749 5036 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:35143 every 8 connection(s)
I20260810 14:45:47.784123 4882 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:45:47.792483 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:47.786660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:36557 (local address 127.24.215.190:41737)
0810 14:45:47.787206 (+ 546us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:47.787210 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:47.788507 (+ 1297us) server_negotiation.cc:407] Connection header received
0810 14:45:47.790211 (+ 1704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:47.790217 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:47.790295 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:47.790414 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:47.791700 (+ 1286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:47.791706 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:47.791711 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:45:47.791732 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:47.791779 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:45:47.791790 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:47.791926 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:47.792124 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 14:45:47.792195 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":238,"threads_started":1}
I20260810 14:45:47.793354 5041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:47.787380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41737 (local address 127.24.215.129:36557)
0810 14:45:47.788332 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:47.788375 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:47.789944 (+ 1569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:47.790660 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:47.790668 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:47.790755 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:47.791402 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:45:47.791404 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:47.791442 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:45:47.791534 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:47.791889 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:47.791902 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:47.791912 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:45:47.792196 (+ 284us) client_negotiation.cc:240] Negotiation successful
0810 14:45:47.792428 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":254,"threads_started":1}
I20260810 14:45:47.793772 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 4882
I20260810 14:45:47.793995 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:45:47.794759 25438 external_mini_cluster.cc:1428] Setting key 31ebb46ecf57857cd60d1d2fe0a04767
I20260810 14:45:47.794891 5037 heartbeater.cc:347] Connected to a master server at 127.24.215.190:41737
I20260810 14:45:47.795234 5037 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:47.795885 5037 heartbeater.cc:511] Master 127.24.215.190:41737 requested a full tablet report, sending...
I20260810 14:45:47.797791 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:41737
--builtin_ntp_servers=127.24.215.148:45939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 14:45:47.798267 4810 ts_manager.cc:194] Registered new tserver with Master: b45f76bc69c2402dbb56122166cb0e79 (127.24.215.129:35143)
W20260810 14:45:48.147318 5044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:48.147610 5044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:48.147800 5044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:48.155259 5044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:48.155391 5044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:48.155561 5044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:45:48.164873 5044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:41737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:48.166241 5044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:48.168093 5044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:48.182090 5051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:48.182092 5053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:48.182557 5050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:48.183169 5044 server_base.cc:1034] running on GCE node
I20260810 14:45:48.183727 5044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:48.184713 5044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:48.185907 5044 hybrid_clock.cc:648] HybridClock initialized: now 1786373148185848 us; error 54 us; skew 500 ppm
I20260810 14:45:48.186226 5044 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:45:48.194067 5044 webserver.cc:460] Webserver started at http://127.24.215.130:34303/ using document root <none> and password file <none>
I20260810 14:45:48.194891 5044 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:48.195014 5044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:48.195382 5044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:48.198345 5044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "7947dd60d3f3435ba97f2706f5fc7741"
format_stamp: "Formatted at 2026-08-10 14:45:48 on dist-test-slave-vkjp"
server_key: "ec9a64a3a5db0c5ac3a3bef9b682058e"
server_key_iv: "255d06bc5b278287c94c70098da644e8"
server_key_version: "encryptionkey@0"
I20260810 14:45:48.199215 5044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "7947dd60d3f3435ba97f2706f5fc7741"
format_stamp: "Formatted at 2026-08-10 14:45:48 on dist-test-slave-vkjp"
server_key: "ec9a64a3a5db0c5ac3a3bef9b682058e"
server_key_iv: "255d06bc5b278287c94c70098da644e8"
server_key_version: "encryptionkey@0"
I20260810 14:45:48.204902 5044 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:45:48.208436 5059 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:48.209865 5044 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:45:48.210021 5044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "7947dd60d3f3435ba97f2706f5fc7741"
format_stamp: "Formatted at 2026-08-10 14:45:48 on dist-test-slave-vkjp"
server_key: "ec9a64a3a5db0c5ac3a3bef9b682058e"
server_key_iv: "255d06bc5b278287c94c70098da644e8"
server_key_version: "encryptionkey@0"
I20260810 14:45:48.210256 5044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:48.216131 5044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:48.216970 5044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:48.217244 5044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:48.218151 5044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:48.219602 5044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:48.219722 5044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:48.219858 5044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:48.219947 5044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:48.256000 5044 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:46359
I20260810 14:45:48.256086 5192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:46359 every 8 connection(s)
I20260810 14:45:48.257377 5044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:45:48.265555 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:48.260053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:58999 (local address 127.24.215.190:41737)
0810 14:45:48.260248 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:48.260253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:48.261592 (+ 1339us) server_negotiation.cc:407] Connection header received
0810 14:45:48.263331 (+ 1739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:48.263337 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:48.263412 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:48.263534 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:48.264788 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:48.264794 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:48.264799 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:45:48.264822 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:48.264861 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:45:48.264873 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:48.264997 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:48.265195 (+ 198us) server_negotiation.cc:299] Negotiation successful
0810 14:45:48.265268 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 14:45:48.266251 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 5044
I20260810 14:45:48.266433 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:45:48.266535 5196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:48.260527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41737 (local address 127.24.215.130:58999)
0810 14:45:48.261427 (+ 900us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:48.261473 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:48.263012 (+ 1539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:48.263762 (+ 750us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:48.263770 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:48.263856 (+ 86us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:48.264498 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:45:48.264501 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:48.264539 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:45:48.264621 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:48.264968 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:48.264991 (+ 23us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:48.264998 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 14:45:48.265278 (+ 280us) client_negotiation.cc:240] Negotiation successful
0810 14:45:48.265527 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":268,"threads_started":1}
I20260810 14:45:48.267086 25438 external_mini_cluster.cc:1428] Setting key c6b04e898ff12670e98994d39ca82fa4
I20260810 14:45:48.268051 5193 heartbeater.cc:347] Connected to a master server at 127.24.215.190:41737
I20260810 14:45:48.268365 5193 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:48.269006 5193 heartbeater.cc:511] Master 127.24.215.190:41737 requested a full tablet report, sending...
I20260810 14:45:48.270074 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:41737
--builtin_ntp_servers=127.24.215.148:45939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260810 14:45:48.270478 4810 ts_manager.cc:194] Registered new tserver with Master: 7947dd60d3f3435ba97f2706f5fc7741 (127.24.215.130:46359)
W20260810 14:45:48.612335 5197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:48.612674 5197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:48.612843 5197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:48.620114 5197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:48.620260 5197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:48.620448 5197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:45:48.629513 5197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:41737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:48.630885 5197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:48.632685 5197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:48.645972 5205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:48.645936 5207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:48.646018 5209 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:48.646899 5197 server_base.cc:1034] running on GCE node
I20260810 14:45:48.647580 5197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:48.648514 5197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:48.649741 5197 hybrid_clock.cc:648] HybridClock initialized: now 1786373148649693 us; error 62 us; skew 500 ppm
I20260810 14:45:48.650059 5197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:45:48.652417 5197 webserver.cc:460] Webserver started at http://127.24.215.131:41629/ using document root <none> and password file <none>
I20260810 14:45:48.653124 5197 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:48.653256 5197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:48.653590 5197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:48.656576 5197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "4edfbb72243e452fb0e79f5565071d95"
format_stamp: "Formatted at 2026-08-10 14:45:48 on dist-test-slave-vkjp"
server_key: "62c27e42bcf50a84620995f5f78e68d1"
server_key_iv: "a997fe223b7b8306aba7dc579b6b6d2b"
server_key_version: "encryptionkey@0"
I20260810 14:45:48.657373 5197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "4edfbb72243e452fb0e79f5565071d95"
format_stamp: "Formatted at 2026-08-10 14:45:48 on dist-test-slave-vkjp"
server_key: "62c27e42bcf50a84620995f5f78e68d1"
server_key_iv: "a997fe223b7b8306aba7dc579b6b6d2b"
server_key_version: "encryptionkey@0"
I20260810 14:45:48.662648 5197 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:45:48.666175 5215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:48.667575 5197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:45:48.667757 5197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "4edfbb72243e452fb0e79f5565071d95"
format_stamp: "Formatted at 2026-08-10 14:45:48 on dist-test-slave-vkjp"
server_key: "62c27e42bcf50a84620995f5f78e68d1"
server_key_iv: "a997fe223b7b8306aba7dc579b6b6d2b"
server_key_version: "encryptionkey@0"
I20260810 14:45:48.668004 5197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:48.673674 5197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:48.674456 5197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:48.674734 5197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:48.675731 5197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:48.677199 5197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:48.677322 5197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:48.677461 5197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:48.677559 5197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:48.711761 5197 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:33139
I20260810 14:45:48.711823 5348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:33139 every 8 connection(s)
I20260810 14:45:48.713102 5197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:45:48.717303 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 5197
I20260810 14:45:48.717490 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:45:48.718071 25438 external_mini_cluster.cc:1428] Setting key 48e8546896df20ae4823bfdfdda442fb
I20260810 14:45:48.722553 5040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:48.715913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:48267 (local address 127.24.215.190:41737)
0810 14:45:48.716146 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:48.716153 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:48.718017 (+ 1864us) server_negotiation.cc:407] Connection header received
0810 14:45:48.720213 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:48.720220 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:48.720282 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:48.720368 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:48.721796 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:48.721802 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:48.721806 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:45:48.721830 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:48.721860 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:45:48.721872 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:48.721991 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:48.722199 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 14:45:48.722265 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:45:48.723496 5351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:48.716679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:41737 (local address 127.24.215.131:48267)
0810 14:45:48.717834 (+ 1155us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:48.717882 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:48.719971 (+ 2089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:48.720637 (+ 666us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:48.720648 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:48.720766 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:48.721511 (+ 745us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:45:48.721514 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:48.721555 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:45:48.721636 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:48.721966 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:48.721984 (+ 18us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:48.721991 (+ 7us) client_negotiation.cc:769] Sending connection context
0810 14:45:48.722281 (+ 290us) client_negotiation.cc:240] Negotiation successful
0810 14:45:48.722522 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":329,"threads_started":1}
I20260810 14:45:48.724745 5349 heartbeater.cc:347] Connected to a master server at 127.24.215.190:41737
I20260810 14:45:48.725001 5349 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:48.725524 5349 heartbeater.cc:511] Master 127.24.215.190:41737 requested a full tablet report, sending...
I20260810 14:45:48.726770 4810 ts_manager.cc:194] Registered new tserver with Master: 4edfbb72243e452fb0e79f5565071d95 (127.24.215.131:33139)
I20260810 14:45:48.733533 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260810 14:45:48.739751 5040 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:45:48.738181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43188 (local address 127.24.215.190:41737)
0810 14:45:48.738341 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:48.738346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:48.738811 (+ 465us) server_negotiation.cc:407] Connection header received
0810 14:45:48.738943 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:48.738947 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:48.738999 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:48.739122 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:48.739524 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43188: BlockingRecv error: recv got EOF from 127.0.0.1:43188 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:45:48.740751 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 4882
I20260810 14:45:48.744772 5359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:48.741746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33068 (local address 127.24.215.129:35143)
0810 14:45:48.742337 (+ 591us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:48.742343 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:48.742370 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:45:48.742430 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:48.742436 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:48.742637 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:48.742759 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:48.743924 (+ 1165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:48.743952 (+ 28us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:48.743971 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:45:48.743996 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:48.744051 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:45:48.744067 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:48.744174 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:48.744326 (+ 152us) server_negotiation.cc:299] Negotiation successful
0810 14:45:48.744441 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":286,"threads_started":1}
I20260810 14:45:48.745982 5030 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:48.827873 5037 heartbeater.cc:503] Master 127.24.215.190:41737 was elected leader, sending a full tablet report...
I20260810 14:45:48.829159 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 4882
W20260810 14:45:48.850302 4791 connection.cc:535] server connection from 127.24.215.129:36557 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 14:45:48.850551 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 5044
I20260810 14:45:48.854389 5361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:48.851668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50188 (local address 127.24.215.130:46359)
0810 14:45:48.852190 (+ 522us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:48.852197 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:48.852225 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:45:48.852287 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:48.852293 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:48.852623 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:48.852756 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:48.853675 (+ 919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:48.853681 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:48.853692 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:45:48.853711 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:48.853751 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:45:48.853766 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:48.853869 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:48.853964 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 14:45:48.854077 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":254,"threads_started":1}
I20260810 14:45:48.855707 5188 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:48.935945 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 5044
I20260810 14:45:48.956018 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 5197
I20260810 14:45:48.960059 5363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:48.957038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55924 (local address 127.24.215.131:33139)
0810 14:45:48.957626 (+ 588us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:48.957633 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:48.957663 (+ 30us) server_negotiation.cc:407] Connection header received
0810 14:45:48.957727 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:48.957733 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:48.957948 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:48.958123 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:48.959298 (+ 1175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:48.959305 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:48.959316 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:45:48.959328 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:48.959370 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:45:48.959386 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:48.959582 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:48.959651 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 14:45:48.959759 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":272,"threads_started":1}
I20260810 14:45:48.961405 5344 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:49.044763 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 5197
I20260810 14:45:49.065466 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 4773
I20260810 14:45:49.066813 4847 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:49.191414 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 4773
2026-08-10T14:45:49Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2461 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:45:49 dist-test-slave-vkjp krb5kdc[5369](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:45:49 dist-test-slave-vkjp krb5kdc[5369](info): set up 2 sockets
Aug 10 14:45:49 dist-test-slave-vkjp krb5kdc[5369](info): commencing operation
krb5kdc: starting...
W20260810 14:45:50.400241 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:45:50 dist-test-slave-vkjp krb5kdc[5369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:45:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:45:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:45:50.668968 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:34303
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:44205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:34303
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:51.005487 5385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:51.005779 5385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:51.005959 5385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:51.013266 5385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:45:51.013382 5385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:51.013466 5385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:51.013576 5385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:45:51.013640 5385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:45:51.023494 5385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:34303
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:34303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:51.024843 5385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:51.026150 5385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:51.038311 5390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:51.038419 5395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:51.038427 5391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:51.039364 5385 server_base.cc:1034] running on GCE node
I20260810 14:45:51.040059 5385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:51.041373 5385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:51.042575 5385 hybrid_clock.cc:648] HybridClock initialized: now 1786373151042521 us; error 48 us; skew 500 ppm
I20260810 14:45:51.042874 5385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:51 dist-test-slave-vkjp krb5kdc[5369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373151, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:51.045964 5385 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:45:51.047510 5385 webserver.cc:460] Webserver started at http://127.24.215.190:34113/ using document root <none> and password file <none>
I20260810 14:45:51.048175 5385 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:51.048316 5385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:51.048645 5385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:51.051800 5385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "7293ae109ebb4deca41ab1466983fdd5"
format_stamp: "Formatted at 2026-08-10 14:45:51 on dist-test-slave-vkjp"
server_key: "945ea516f1b8f756e8aae6b67c1da6f4"
server_key_iv: "6a888b354c2667a75fffabc64a00995c"
server_key_version: "encryptionkey@0"
I20260810 14:45:51.052673 5385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "7293ae109ebb4deca41ab1466983fdd5"
format_stamp: "Formatted at 2026-08-10 14:45:51 on dist-test-slave-vkjp"
server_key: "945ea516f1b8f756e8aae6b67c1da6f4"
server_key_iv: "6a888b354c2667a75fffabc64a00995c"
server_key_version: "encryptionkey@0"
I20260810 14:45:51.057885 5385 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:45:51.061456 5402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:51.062752 5385 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 14:45:51.062942 5385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "7293ae109ebb4deca41ab1466983fdd5"
format_stamp: "Formatted at 2026-08-10 14:45:51 on dist-test-slave-vkjp"
server_key: "945ea516f1b8f756e8aae6b67c1da6f4"
server_key_iv: "6a888b354c2667a75fffabc64a00995c"
server_key_version: "encryptionkey@0"
I20260810 14:45:51.063235 5385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:51.077805 5385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:51.081940 5385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:51.082237 5385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:51.100690 5385 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:34303
I20260810 14:45:51.100725 5463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:34303 every 8 connection(s)
I20260810 14:45:51.102078 5385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:45:51.104132 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 5385
I20260810 14:45:51.104341 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:45:51.105003 25438 external_mini_cluster.cc:1428] Setting key be748f3cdb92dd7cc280cc9c56378cde
I20260810 14:45:51.105942 5465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 14:45:51 dist-test-slave-vkjp krb5kdc[5369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:45:51.120157 5465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5: Bootstrap starting.
I20260810 14:45:51.124257 5465 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:51.125526 5465 log.cc:824] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:51.129088 5465 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5: No bootstrap required, opened a new log
I20260810 14:45:51.129460 5468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:51.107154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59806 (local address 127.24.215.190:34303)
0810 14:45:51.107978 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:51.107995 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:51.108044 (+ 49us) server_negotiation.cc:407] Connection header received
0810 14:45:51.109715 (+ 1671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:51.109744 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:51.110277 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:51.110736 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:51.112188 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:51.114186 (+ 1998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:51.115568 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:51.116593 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:51.116787 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:51.120916 (+ 4129us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:51.120930 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:51.120947 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:51.120996 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:51.124997 (+ 4001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:51.125711 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:51.125720 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:51.125727 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:51.125931 (+ 204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:51.126470 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:51.126480 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:51.126482 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:51.127043 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:51.127396 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:51.127751 (+ 355us) server_negotiation.cc:299] Negotiation successful
0810 14:45:51.128160 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":583,"thread_start_us":337,"threads_started":1}
I20260810 14:45:51.137401 5465 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7293ae109ebb4deca41ab1466983fdd5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34303 } }
I20260810 14:45:51.137737 5465 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:51.137801 5465 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7293ae109ebb4deca41ab1466983fdd5, State: Initialized, Role: FOLLOWER
I20260810 14:45:51.138305 5465 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7293ae109ebb4deca41ab1466983fdd5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34303 } }
I20260810 14:45:51.138507 5465 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:45:51.138607 5465 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:45:51.138725 5465 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:51.140894 5465 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7293ae109ebb4deca41ab1466983fdd5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34303 } }
I20260810 14:45:51.141412 5465 leader_election.cc:302] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7293ae109ebb4deca41ab1466983fdd5; no voters:
I20260810 14:45:51.142166 5465 leader_election.cc:288] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:45:51.142299 5472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:45:51.143023 5472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [term 1 LEADER]: Becoming Leader. State: Replica: 7293ae109ebb4deca41ab1466983fdd5, State: Running, Role: LEADER
I20260810 14:45:51.143637 5472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7293ae109ebb4deca41ab1466983fdd5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34303 } }
I20260810 14:45:51.144323 5465 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:45:51.146456 5473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7293ae109ebb4deca41ab1466983fdd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7293ae109ebb4deca41ab1466983fdd5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34303 } } }
I20260810 14:45:51.146914 5473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:51.146481 5474 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7293ae109ebb4deca41ab1466983fdd5. Latest consensus state: current_term: 1 leader_uuid: "7293ae109ebb4deca41ab1466983fdd5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7293ae109ebb4deca41ab1466983fdd5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34303 } } }
I20260810 14:45:51.149101 5474 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5 [sys.catalog]: This master's current role is: LEADER
I20260810 14:45:51.150816 5481 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:45:51.156323 5481 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:45:51.165560 5481 catalog_manager.cc:1347] Generated new cluster ID: feeb0c0702b8460fa742a9076a692cea
I20260810 14:45:51.165694 5481 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:45:51.183555 5481 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:45:51.184859 5481 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:45:51.193176 5481 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5: Generated new TSK 0
I20260810 14:45:51.194254 5481 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:45:51.270825 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34303
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:44205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:51.618538 5497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:51.618816 5497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:51.619000 5497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:51.626333 5497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:51.626438 5497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:51.626642 5497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:45:51.636983 5497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:51.638382 5497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:51.639735 5497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:51.652853 5502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:51.652889 5503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:51.652982 5505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:51.656581 5497 server_base.cc:1034] running on GCE node
I20260810 14:45:51.657192 5497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:51.658242 5497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:51.659471 5497 hybrid_clock.cc:648] HybridClock initialized: now 1786373151659417 us; error 59 us; skew 500 ppm
I20260810 14:45:51.659777 5497 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:51 dist-test-slave-vkjp krb5kdc[5369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373151, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:51.662949 5497 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:45:51.664510 5497 webserver.cc:460] Webserver started at http://127.24.215.129:36179/ using document root <none> and password file <none>
I20260810 14:45:51.665212 5497 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:51.665352 5497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:51.665656 5497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:51.668882 5497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "500cf995c6e64e9c84149212bec95c38"
format_stamp: "Formatted at 2026-08-10 14:45:51 on dist-test-slave-vkjp"
server_key: "b74d576cc321730cf28b4cb876df4f00"
server_key_iv: "8656002f11cad97e5f154e8dcd4db18a"
server_key_version: "encryptionkey@0"
I20260810 14:45:51.669677 5497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "500cf995c6e64e9c84149212bec95c38"
format_stamp: "Formatted at 2026-08-10 14:45:51 on dist-test-slave-vkjp"
server_key: "b74d576cc321730cf28b4cb876df4f00"
server_key_iv: "8656002f11cad97e5f154e8dcd4db18a"
server_key_version: "encryptionkey@0"
I20260810 14:45:51.674820 5497 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 14:45:51.678293 5513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:51.679661 5497 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:45:51.679833 5497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "500cf995c6e64e9c84149212bec95c38"
format_stamp: "Formatted at 2026-08-10 14:45:51 on dist-test-slave-vkjp"
server_key: "b74d576cc321730cf28b4cb876df4f00"
server_key_iv: "8656002f11cad97e5f154e8dcd4db18a"
server_key_version: "encryptionkey@0"
I20260810 14:45:51.680096 5497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:51.722229 5497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:51.726418 5497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:51.726730 5497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:51.727818 5497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:51.729229 5497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:51.729332 5497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:51.729480 5497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:51.729552 5497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:51.762760 5497 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:46831
I20260810 14:45:51.762807 5653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:46831 every 8 connection(s)
I20260810 14:45:51.764072 5497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:45:51.768272 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 5497
I20260810 14:45:51.768519 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:45:51.769451 25438 external_mini_cluster.cc:1428] Setting key 9d677d46e90b5926d8a166925cf5652a
Aug 10 14:45:51 dist-test-slave-vkjp krb5kdc[5369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373151, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:45:51.786707 5656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:51.766569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:46663 (local address 127.24.215.190:34303)
0810 14:45:51.767059 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:51.767067 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:51.768831 (+ 1764us) server_negotiation.cc:407] Connection header received
0810 14:45:51.771070 (+ 2239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:51.771075 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:51.771188 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:51.771298 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:51.772872 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:51.774057 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:51.775770 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:51.776460 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:51.776619 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:51.780218 (+ 3599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:51.780227 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:51.780235 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:51.780278 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:51.783380 (+ 3102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:51.784288 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:51.784292 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:51.784295 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:51.784433 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:51.784968 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:51.784986 (+ 18us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:51.784989 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:51.785259 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:51.785413 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:51.786121 (+ 708us) server_negotiation.cc:299] Negotiation successful
0810 14:45:51.786350 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":240,"threads_started":1}
I20260810 14:45:51.787714 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:51.767247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34303 (local address 127.24.215.129:46663)
0810 14:45:51.768641 (+ 1394us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:51.768699 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:51.770796 (+ 2097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:51.771600 (+ 804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:51.771609 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:51.772121 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:51.772691 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:51.772708 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:51.774234 (+ 1526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:51.774238 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:51.775617 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:51.775630 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:51.776679 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:51.776684 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:51.776877 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:51.777584 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:51.777609 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:51.779994 (+ 2385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:51.783510 (+ 3516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:51.783522 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:51.783541 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:51.784165 (+ 624us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:51.784586 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:51.784589 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:51.784592 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:51.784840 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:51.785388 (+ 548us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:51.785396 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:51.785724 (+ 328us) client_negotiation.cc:769] Sending connection context
0810 14:45:51.785992 (+ 268us) client_negotiation.cc:240] Negotiation successful
0810 14:45:51.786514 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":259,"threads_started":1}
I20260810 14:45:51.789326 5654 heartbeater.cc:347] Connected to a master server at 127.24.215.190:34303
I20260810 14:45:51.789700 5654 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:51.790660 5654 heartbeater.cc:511] Master 127.24.215.190:34303 requested a full tablet report, sending...
I20260810 14:45:51.793256 5421 ts_manager.cc:194] Registered new tserver with Master: 500cf995c6e64e9c84149212bec95c38 (127.24.215.129:46831)
I20260810 14:45:51.795603 5421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:46663
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:45:51.830785 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34303
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:44205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:52.165937 5661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:52.166260 5661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:52.166411 5661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:52.173875 5661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:52.173995 5661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:52.174109 5661 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:45:52.184012 5661 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:52.185312 5661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:52.186568 5661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:52.199487 5668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:52.199817 5671 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:52.199841 5667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:52.200515 5661 server_base.cc:1034] running on GCE node
I20260810 14:45:52.201063 5661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:52.201920 5661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:52.203172 5661 hybrid_clock.cc:648] HybridClock initialized: now 1786373152203119 us; error 32 us; skew 500 ppm
I20260810 14:45:52.203490 5661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:52 dist-test-slave-vkjp krb5kdc[5369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373152, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:52.206621 5661 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:45:52.208164 5661 webserver.cc:460] Webserver started at http://127.24.215.130:36115/ using document root <none> and password file <none>
I20260810 14:45:52.208873 5661 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:52.209038 5661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:52.209378 5661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:52.212740 5661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "19b17aad46894363bd2ba52127894c43"
format_stamp: "Formatted at 2026-08-10 14:45:52 on dist-test-slave-vkjp"
server_key: "30a97b99671a22c702011afe9dc79409"
server_key_iv: "06e3903c9d6a2a43cb8e209436b4dc01"
server_key_version: "encryptionkey@0"
I20260810 14:45:52.213613 5661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "19b17aad46894363bd2ba52127894c43"
format_stamp: "Formatted at 2026-08-10 14:45:52 on dist-test-slave-vkjp"
server_key: "30a97b99671a22c702011afe9dc79409"
server_key_iv: "06e3903c9d6a2a43cb8e209436b4dc01"
server_key_version: "encryptionkey@0"
I20260810 14:45:52.218719 5661 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 14:45:52.222191 5681 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:52.223577 5661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:45:52.223747 5661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "19b17aad46894363bd2ba52127894c43"
format_stamp: "Formatted at 2026-08-10 14:45:52 on dist-test-slave-vkjp"
server_key: "30a97b99671a22c702011afe9dc79409"
server_key_iv: "06e3903c9d6a2a43cb8e209436b4dc01"
server_key_version: "encryptionkey@0"
I20260810 14:45:52.223950 5661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:52.249939 5661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:52.254091 5661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:52.254408 5661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:52.255519 5661 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:52.256835 5661 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:52.256939 5661 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:52.257091 5661 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:52.257164 5661 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:52.290808 5661 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:41827
I20260810 14:45:52.290843 5813 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:41827 every 8 connection(s)
I20260810 14:45:52.292142 5661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:45:52.296540 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 5661
I20260810 14:45:52.296728 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:45:52.297315 25438 external_mini_cluster.cc:1428] Setting key 1a8351b34d3008ed282b30d4b7edbe23
Aug 10 14:45:52 dist-test-slave-vkjp krb5kdc[5369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373152, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:45:52.315675 5816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:52.294647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:52375 (local address 127.24.215.190:34303)
0810 14:45:52.295158 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:52.295165 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:52.296102 (+ 937us) server_negotiation.cc:407] Connection header received
0810 14:45:52.298602 (+ 2500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:52.298608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:52.298689 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:52.298836 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:52.300795 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.301553 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.304119 (+ 2566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.304586 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.304699 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:52.309107 (+ 4408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:52.309112 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:52.309116 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:52.309154 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:52.312247 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:52.313077 (+ 830us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:52.313082 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:52.313085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:52.313223 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:52.313711 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:52.313716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:52.313718 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:52.314005 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:52.314180 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:52.314961 (+ 781us) server_negotiation.cc:299] Negotiation successful
0810 14:45:52.315242 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":232,"threads_started":1}
I20260810 14:45:52.316766 5817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:52.295271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34303 (local address 127.24.215.130:52375)
0810 14:45:52.295948 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:52.295986 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:52.298291 (+ 2305us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:52.299003 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:52.299011 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:52.299742 (+ 731us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:52.300572 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:52.300602 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.301719 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.301728 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:52.303932 (+ 2204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:52.303951 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.304821 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.304830 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:52.305164 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:52.305971 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:52.306007 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:52.308806 (+ 2799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:52.312420 (+ 3614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:52.312430 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:52.312449 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:52.312923 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:52.313348 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:52.313354 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:52.313358 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:52.313568 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:52.314160 (+ 592us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:52.314168 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:52.314545 (+ 377us) client_negotiation.cc:769] Sending connection context
0810 14:45:52.314844 (+ 299us) client_negotiation.cc:240] Negotiation successful
0810 14:45:52.315477 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":267,"threads_started":1}
I20260810 14:45:52.318429 5814 heartbeater.cc:347] Connected to a master server at 127.24.215.190:34303
I20260810 14:45:52.318774 5814 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:52.319759 5814 heartbeater.cc:511] Master 127.24.215.190:34303 requested a full tablet report, sending...
I20260810 14:45:52.321522 5421 ts_manager.cc:194] Registered new tserver with Master: 19b17aad46894363bd2ba52127894c43 (127.24.215.130:41827)
I20260810 14:45:52.322443 5421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:52375
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:45:52.360510 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34303
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:44205
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:45:52.702201 5820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:45:52.702484 5820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:45:52.702581 5820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:45:52.709957 5820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:45:52.710070 5820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:45:52.710186 5820 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:45:52.719866 5820 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44205
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34303
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:45:52.721153 5820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:45:52.722586 5820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:45:52.736958 5827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:52.736922 5828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:45:52.736908 5830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:45:52.737926 5820 server_base.cc:1034] running on GCE node
I20260810 14:45:52.738638 5820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:45:52.739799 5820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:45:52.741055 5820 hybrid_clock.cc:648] HybridClock initialized: now 1786373152740986 us; error 38 us; skew 500 ppm
I20260810 14:45:52.741468 5820 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:45:52 dist-test-slave-vkjp krb5kdc[5369](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373152, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:45:52.745086 5820 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:45:52.746845 5820 webserver.cc:460] Webserver started at http://127.24.215.131:37913/ using document root <none> and password file <none>
I20260810 14:45:52.747654 5820 fs_manager.cc:359] Metadata directory not provided
I20260810 14:45:52.747792 5820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:45:52.748093 5820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:45:52.751603 5820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "0b7f46fe1f6f46fc9a453321dd40b82f"
format_stamp: "Formatted at 2026-08-10 14:45:52 on dist-test-slave-vkjp"
server_key: "02f1ec4d05af6ae3dd22437579694175"
server_key_iv: "9177e4c160043ccead08778c02f90987"
server_key_version: "encryptionkey@0"
I20260810 14:45:52.752570 5820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "0b7f46fe1f6f46fc9a453321dd40b82f"
format_stamp: "Formatted at 2026-08-10 14:45:52 on dist-test-slave-vkjp"
server_key: "02f1ec4d05af6ae3dd22437579694175"
server_key_iv: "9177e4c160043ccead08778c02f90987"
server_key_version: "encryptionkey@0"
I20260810 14:45:52.759207 5820 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.001s
I20260810 14:45:52.763064 5837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:52.764581 5820 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:45:52.764755 5820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "0b7f46fe1f6f46fc9a453321dd40b82f"
format_stamp: "Formatted at 2026-08-10 14:45:52 on dist-test-slave-vkjp"
server_key: "02f1ec4d05af6ae3dd22437579694175"
server_key_iv: "9177e4c160043ccead08778c02f90987"
server_key_version: "encryptionkey@0"
I20260810 14:45:52.764950 5820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:45:52.778447 5820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:45:52.782352 5820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:45:52.782656 5820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:45:52.783766 5820 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:45:52.785198 5820 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:45:52.785301 5820 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:52.785457 5820 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:45:52.785539 5820 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:45:52.799362 5654 heartbeater.cc:503] Master 127.24.215.190:34303 was elected leader, sending a full tablet report...
I20260810 14:45:52.819729 5820 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:32989
I20260810 14:45:52.819774 5977 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:32989 every 8 connection(s)
I20260810 14:45:52.821053 5820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:45:52.827454 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 5820
I20260810 14:45:52.827648 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:45:52.828102 25438 external_mini_cluster.cc:1428] Setting key 28dbc6672f8540c9f708695f53436b5f
Aug 10 14:45:52 dist-test-slave-vkjp krb5kdc[5369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373152, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:45:52.841863 5980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:52.823691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:35515 (local address 127.24.215.190:34303)
0810 14:45:52.824187 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:52.824194 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:52.825164 (+ 970us) server_negotiation.cc:407] Connection header received
0810 14:45:52.826970 (+ 1806us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:52.826976 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:52.827127 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:52.827256 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:52.829065 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.830202 (+ 1137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.831954 (+ 1752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.832340 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.832447 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:52.835770 (+ 3323us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:52.835776 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:52.835780 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:52.835828 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:52.838940 (+ 3112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:52.839656 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:52.839661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:52.839663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:52.839811 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:52.840194 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:52.840198 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:52.840200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:52.840480 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:52.840606 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:52.841319 (+ 713us) server_negotiation.cc:299] Negotiation successful
0810 14:45:52.841485 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":221,"threads_started":1}
I20260810 14:45:52.843142 5981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:52.824288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34303 (local address 127.24.215.131:35515)
0810 14:45:52.825005 (+ 717us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:52.825049 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:52.826673 (+ 1624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:52.827505 (+ 832us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:52.827513 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:52.828170 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:45:52.828876 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:52.828906 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.830352 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.830356 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:52.831827 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:52.831839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.832536 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.832540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:52.832724 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:52.833347 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:45:52.833369 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:45:52.835578 (+ 2209us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:45:52.839121 (+ 3543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:52.839130 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:52.839149 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:52.839546 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:52.839909 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:45:52.839912 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:45:52.839914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:45:52.840091 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:45:52.840586 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:45:52.840591 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:45:52.840906 (+ 315us) client_negotiation.cc:769] Sending connection context
0810 14:45:52.841171 (+ 265us) client_negotiation.cc:240] Negotiation successful
0810 14:45:52.841677 (+ 506us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":271,"threads_started":1}
I20260810 14:45:52.844587 5978 heartbeater.cc:347] Connected to a master server at 127.24.215.190:34303
I20260810 14:45:52.844938 5978 heartbeater.cc:464] Registering TS with master...
I20260810 14:45:52.845770 5978 heartbeater.cc:511] Master 127.24.215.190:34303 requested a full tablet report, sending...
I20260810 14:45:52.847258 5421 ts_manager.cc:194] Registered new tserver with Master: 0b7f46fe1f6f46fc9a453321dd40b82f (127.24.215.131:32989)
I20260810 14:45:52.848073 5421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:35515
I20260810 14:45:52.855746 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:45:52.873034 5980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:52.860302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59810 (local address 127.24.215.190:34303)
0810 14:45:52.860466 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:52.860470 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:52.860931 (+ 461us) server_negotiation.cc:407] Connection header received
0810 14:45:52.861098 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:52.861102 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:52.861175 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:52.861254 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:52.862596 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.863383 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.864571 (+ 1188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.864997 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.865091 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:52.867432 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:52.867436 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:52.867440 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:52.867470 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:52.870381 (+ 2911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:52.871034 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:52.871041 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:52.871045 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:52.871220 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:52.871577 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:52.871583 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:52.871587 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:52.871901 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:52.872155 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:52.872503 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 14:45:52.872724 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 14:45:52.877157 5421 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:45:52.884317 5421 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:45:52.913735 5994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:52.896956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46831 (local address 127.0.0.1:52986)
0810 14:45:52.898762 (+ 1806us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:52.898804 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:52.898992 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:52.901577 (+ 2585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:52.901589 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:52.901630 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:52.902129 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:52.902145 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.906230 (+ 4085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.906240 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:52.910644 (+ 4404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:52.910661 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.912745 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.912751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:52.912937 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:52.912987 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 14:45:52.913112 (+ 125us) client_negotiation.cc:240] Negotiation successful
0810 14:45:52.913324 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1581,"thread_start_us":13940,"threads_started":1}
I20260810 14:45:52.914841 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:52.897586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52986 (local address 127.24.215.129:46831)
0810 14:45:52.898248 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:52.898260 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:52.899235 (+ 975us) server_negotiation.cc:407] Connection header received
0810 14:45:52.899388 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:52.899399 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:52.899684 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:52.901213 (+ 1529us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:52.902408 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.906001 (+ 3593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.910862 (+ 4861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.912504 (+ 1642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.912721 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:52.912905 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:52.914257 (+ 1352us) server_negotiation.cc:299] Negotiation successful
0810 14:45:52.914409 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":657,"threads_started":1}
I20260810 14:45:52.918771 5992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:52.897711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:32989 (local address 127.0.0.1:33050)
0810 14:45:52.911176 (+ 13465us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:52.911196 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:52.911341 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:52.911991 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:52.911995 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:52.912023 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:52.912484 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:52.912497 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.914609 (+ 2112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.914612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:52.916539 (+ 1927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:52.916554 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.917902 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.917908 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:52.918077 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:52.918120 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 14:45:52.918234 (+ 114us) client_negotiation.cc:240] Negotiation successful
0810 14:45:52.918367 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13332,"mutex_wait_us":36,"thread_start_us":12704,"threads_started":1}
I20260810 14:45:52.918805 5999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:52.898425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33050 (local address 127.24.215.131:32989)
0810 14:45:52.908799 (+ 10374us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:52.908809 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:52.911307 (+ 2498us) server_negotiation.cc:407] Connection header received
0810 14:45:52.911529 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:52.911535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:52.911737 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:52.911959 (+ 222us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:52.912682 (+ 723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.914421 (+ 1739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.916729 (+ 2308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.917690 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.917906 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:52.918080 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:52.918288 (+ 208us) server_negotiation.cc:299] Negotiation successful
0810 14:45:52.918452 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10184,"thread_start_us":10013,"threads_started":1}
I20260810 14:45:52.919406 5572 tablet_service.cc:1467] Processing CreateTablet for tablet d3e6a100a7d4422792804964cbc7f80a (DEFAULT_TABLE table=test-table [id=d177b0ebb6a1429d8a85a75754e002c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:52.920486 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:52.897208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:41827 (local address 127.0.0.1:38356)
0810 14:45:52.910760 (+ 13552us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:52.910789 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:52.910914 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:52.911714 (+ 800us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:52.911719 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:52.911743 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:52.912225 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:52.912239 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.914540 (+ 2301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.914547 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:52.916701 (+ 2154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:52.916717 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.918840 (+ 2123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.918847 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:52.919638 (+ 791us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:52.919669 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 14:45:52.919866 (+ 197us) client_negotiation.cc:240] Negotiation successful
0810 14:45:52.920089 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13391,"mutex_wait_us":2,"thread_start_us":1229,"threads_started":1}
I20260810 14:45:52.921059 5572 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e6a100a7d4422792804964cbc7f80a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:52.921309 5998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:52.898091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38356 (local address 127.24.215.130:41827)
0810 14:45:52.910104 (+ 12013us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:52.910112 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:52.911097 (+ 985us) server_negotiation.cc:407] Connection header received
0810 14:45:52.911204 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:52.911213 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:52.911507 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:52.911711 (+ 204us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:52.912402 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.914371 (+ 1969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.916872 (+ 2501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:52.918289 (+ 1417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:52.918525 (+ 236us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:52.918680 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:52.920705 (+ 2025us) server_negotiation.cc:299] Negotiation successful
0810 14:45:52.920894 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11845,"thread_start_us":12021,"threads_started":1}
I20260810 14:45:52.922941 5895 tablet_service.cc:1467] Processing CreateTablet for tablet d3e6a100a7d4422792804964cbc7f80a (DEFAULT_TABLE table=test-table [id=d177b0ebb6a1429d8a85a75754e002c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:52.924448 5895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e6a100a7d4422792804964cbc7f80a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:52.925006 5733 tablet_service.cc:1467] Processing CreateTablet for tablet d3e6a100a7d4422792804964cbc7f80a (DEFAULT_TABLE table=test-table [id=d177b0ebb6a1429d8a85a75754e002c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:45:52.926501 5733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3e6a100a7d4422792804964cbc7f80a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:45:52.934273 6000 tablet_bootstrap.cc:492] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38: Bootstrap starting.
I20260810 14:45:52.939064 6000 tablet_bootstrap.cc:654] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:52.939796 6001 tablet_bootstrap.cc:492] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f: Bootstrap starting.
I20260810 14:45:52.941089 6002 tablet_bootstrap.cc:492] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43: Bootstrap starting.
I20260810 14:45:52.941280 6000 log.cc:824] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:52.943740 6001 tablet_bootstrap.cc:654] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:52.944396 6002 tablet_bootstrap.cc:654] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43: Neither blocks nor log segments found. Creating new log.
I20260810 14:45:52.945142 6001 log.cc:824] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:52.945397 6000 tablet_bootstrap.cc:492] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38: No bootstrap required, opened a new log
I20260810 14:45:52.945739 6000 ts_tablet_manager.cc:1397] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38: Time spent bootstrapping tablet: real 0.012s user 0.001s sys 0.005s
I20260810 14:45:52.945693 6002 log.cc:824] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43: Log is configured to *not* fsync() on all Append() calls
I20260810 14:45:52.949261 6002 tablet_bootstrap.cc:492] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43: No bootstrap required, opened a new log
I20260810 14:45:52.949334 6001 tablet_bootstrap.cc:492] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f: No bootstrap required, opened a new log
I20260810 14:45:52.949559 6002 ts_tablet_manager.cc:1397] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 14:45:52.949587 6001 ts_tablet_manager.cc:1397] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260810 14:45:52.956841 6002 raft_consensus.cc:348] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500cf995c6e64e9c84149212bec95c38" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46831 } } peers { permanent_uuid: "19b17aad46894363bd2ba52127894c43" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41827 } } peers { permanent_uuid: "0b7f46fe1f6f46fc9a453321dd40b82f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 32989 } }
I20260810 14:45:52.957268 6002 raft_consensus.cc:374] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:52.957373 6002 raft_consensus.cc:729] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19b17aad46894363bd2ba52127894c43, State: Initialized, Role: FOLLOWER
I20260810 14:45:52.957904 6000 raft_consensus.cc:348] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500cf995c6e64e9c84149212bec95c38" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46831 } } peers { permanent_uuid: "19b17aad46894363bd2ba52127894c43" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41827 } } peers { permanent_uuid: "0b7f46fe1f6f46fc9a453321dd40b82f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 32989 } }
I20260810 14:45:52.958326 6000 raft_consensus.cc:374] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:52.958159 6002 consensus_queue.cc:260] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500cf995c6e64e9c84149212bec95c38" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46831 } } peers { permanent_uuid: "19b17aad46894363bd2ba52127894c43" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41827 } } peers { permanent_uuid: "0b7f46fe1f6f46fc9a453321dd40b82f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 32989 } }
I20260810 14:45:52.958441 6000 raft_consensus.cc:729] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 500cf995c6e64e9c84149212bec95c38, State: Initialized, Role: FOLLOWER
I20260810 14:45:52.959069 6000 consensus_queue.cc:260] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500cf995c6e64e9c84149212bec95c38" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46831 } } peers { permanent_uuid: "19b17aad46894363bd2ba52127894c43" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41827 } } peers { permanent_uuid: "0b7f46fe1f6f46fc9a453321dd40b82f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 32989 } }
I20260810 14:45:52.959658 5814 heartbeater.cc:503] Master 127.24.215.190:34303 was elected leader, sending a full tablet report...
I20260810 14:45:52.960515 6001 raft_consensus.cc:348] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500cf995c6e64e9c84149212bec95c38" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46831 } } peers { permanent_uuid: "19b17aad46894363bd2ba52127894c43" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41827 } } peers { permanent_uuid: "0b7f46fe1f6f46fc9a453321dd40b82f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 32989 } }
I20260810 14:45:52.960876 6002 ts_tablet_manager.cc:1428] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260810 14:45:52.961153 6001 raft_consensus.cc:374] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:45:52.961321 6001 raft_consensus.cc:729] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b7f46fe1f6f46fc9a453321dd40b82f, State: Initialized, Role: FOLLOWER
I20260810 14:45:52.962049 6001 consensus_queue.cc:260] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500cf995c6e64e9c84149212bec95c38" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46831 } } peers { permanent_uuid: "19b17aad46894363bd2ba52127894c43" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41827 } } peers { permanent_uuid: "0b7f46fe1f6f46fc9a453321dd40b82f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 32989 } }
I20260810 14:45:52.963371 5978 heartbeater.cc:503] Master 127.24.215.190:34303 was elected leader, sending a full tablet report...
I20260810 14:45:52.964190 6001 ts_tablet_manager.cc:1428] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f: Time spent starting tablet: real 0.014s user 0.010s sys 0.005s
I20260810 14:45:52.965821 6000 ts_tablet_manager.cc:1428] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38: Time spent starting tablet: real 0.020s user 0.009s sys 0.010s
W20260810 14:45:53.016577 5655 tablet.cc:2369] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:45:53.044307 5815 tablet.cc:2369] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:45:53.072712 5979 tablet.cc:2369] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:45:53.185024 6010 raft_consensus.cc:482] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:45:53.185356 6010 raft_consensus.cc:504] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500cf995c6e64e9c84149212bec95c38" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46831 } } peers { permanent_uuid: "19b17aad46894363bd2ba52127894c43" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41827 } } peers { permanent_uuid: "0b7f46fe1f6f46fc9a453321dd40b82f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 32989 } }
I20260810 14:45:53.187223 6010 leader_election.cc:288] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 500cf995c6e64e9c84149212bec95c38 (127.24.215.129:46831), 19b17aad46894363bd2ba52127894c43 (127.24.215.130:41827)
I20260810 14:45:53.193830 5998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:53.187736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:51387 (local address 127.24.215.130:41827)
0810 14:45:53.188060 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:53.188068 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:53.188092 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:45:53.188182 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:53.188189 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:53.188276 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:53.188392 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:53.188842 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:53.189901 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:53.192104 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:53.192918 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:53.193086 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:53.193160 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:53.193436 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 14:45:53.193557 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
I20260810 14:45:53.193852 5981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:53.187534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:41827 (local address 127.24.215.131:51387)
0810 14:45:53.187855 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:53.187879 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:53.188035 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:53.188419 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:53.188422 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:53.188449 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:53.188687 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:53.188700 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:53.190041 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:53.190047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:53.191891 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:53.191904 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:53.193085 (+ 1181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:53.193091 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:53.193282 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:53.193305 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 14:45:53.193407 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 14:45:53.193538 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":19}
I20260810 14:45:53.194526 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:53.187605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:44923 (local address 127.24.215.129:46831)
0810 14:45:53.187913 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:53.187919 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:53.188575 (+ 656us) server_negotiation.cc:407] Connection header received
0810 14:45:53.188747 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:53.188754 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:53.188840 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:53.188916 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:45:53.189550 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:53.190487 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:53.192771 (+ 2284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:53.193539 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:53.193745 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:53.193829 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:53.194170 (+ 341us) server_negotiation.cc:299] Negotiation successful
0810 14:45:53.194310 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260810 14:45:53.194679 6011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:53.187645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46831 (local address 127.24.215.131:44923)
0810 14:45:53.188461 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0810 14:45:53.188479 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:45:53.188608 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:45:53.188970 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:45:53.188974 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:45:53.189000 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:45:53.189384 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:53.189397 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:53.190687 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:53.190695 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:53.192617 (+ 1922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:45:53.192629 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:45:53.193769 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:45:53.193775 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:45:53.193995 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:53.194041 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 14:45:53.194127 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 14:45:53.194252 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":2,"thread_start_us":444,"threads_started":1}
I20260810 14:45:53.194908 5759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e6a100a7d4422792804964cbc7f80a" candidate_uuid: "0b7f46fe1f6f46fc9a453321dd40b82f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b17aad46894363bd2ba52127894c43" is_pre_election: true
I20260810 14:45:53.195516 5759 raft_consensus.cc:2454] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b7f46fe1f6f46fc9a453321dd40b82f in term 0.
I20260810 14:45:53.195729 5595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e6a100a7d4422792804964cbc7f80a" candidate_uuid: "0b7f46fe1f6f46fc9a453321dd40b82f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500cf995c6e64e9c84149212bec95c38" is_pre_election: true
I20260810 14:45:53.196254 5595 raft_consensus.cc:2454] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b7f46fe1f6f46fc9a453321dd40b82f in term 0.
I20260810 14:45:53.196238 5839 leader_election.cc:302] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b7f46fe1f6f46fc9a453321dd40b82f, 19b17aad46894363bd2ba52127894c43; no voters:
I20260810 14:45:53.196717 6010 raft_consensus.cc:2781] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:45:53.196938 6010 raft_consensus.cc:482] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:45:53.197074 6010 raft_consensus.cc:3037] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:53.199371 6010 raft_consensus.cc:504] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500cf995c6e64e9c84149212bec95c38" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46831 } } peers { permanent_uuid: "19b17aad46894363bd2ba52127894c43" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41827 } } peers { permanent_uuid: "0b7f46fe1f6f46fc9a453321dd40b82f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 32989 } }
I20260810 14:45:53.200001 6010 leader_election.cc:288] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [CANDIDATE]: Term 1 election: Requested vote from peers 500cf995c6e64e9c84149212bec95c38 (127.24.215.129:46831), 19b17aad46894363bd2ba52127894c43 (127.24.215.130:41827)
I20260810 14:45:53.200500 5759 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e6a100a7d4422792804964cbc7f80a" candidate_uuid: "0b7f46fe1f6f46fc9a453321dd40b82f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "19b17aad46894363bd2ba52127894c43"
I20260810 14:45:53.200521 5595 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3e6a100a7d4422792804964cbc7f80a" candidate_uuid: "0b7f46fe1f6f46fc9a453321dd40b82f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "500cf995c6e64e9c84149212bec95c38"
I20260810 14:45:53.200716 5759 raft_consensus.cc:3037] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:53.200718 5595 raft_consensus.cc:3037] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:45:53.202956 5595 raft_consensus.cc:2454] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b7f46fe1f6f46fc9a453321dd40b82f in term 1.
I20260810 14:45:53.202977 5759 raft_consensus.cc:2454] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b7f46fe1f6f46fc9a453321dd40b82f in term 1.
I20260810 14:45:53.203426 5841 leader_election.cc:302] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b7f46fe1f6f46fc9a453321dd40b82f, 500cf995c6e64e9c84149212bec95c38; no voters:
I20260810 14:45:53.203799 6010 raft_consensus.cc:2781] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:45:53.204280 6010 raft_consensus.cc:686] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [term 1 LEADER]: Becoming Leader. State: Replica: 0b7f46fe1f6f46fc9a453321dd40b82f, State: Running, Role: LEADER
I20260810 14:45:53.204777 6010 consensus_queue.cc:237] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500cf995c6e64e9c84149212bec95c38" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46831 } } peers { permanent_uuid: "19b17aad46894363bd2ba52127894c43" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41827 } } peers { permanent_uuid: "0b7f46fe1f6f46fc9a453321dd40b82f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 32989 } }
I20260810 14:45:53.209666 5420 catalog_manager.cc:5495] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b7f46fe1f6f46fc9a453321dd40b82f (127.24.215.131). New cstate: current_term: 1 leader_uuid: "0b7f46fe1f6f46fc9a453321dd40b82f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "500cf995c6e64e9c84149212bec95c38" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46831 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "19b17aad46894363bd2ba52127894c43" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41827 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0b7f46fe1f6f46fc9a453321dd40b82f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 32989 } health_report { overall_health: HEALTHY } } }
I20260810 14:45:53.276106 5999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:53.270430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33066 (local address 127.24.215.131:32989)
0810 14:45:53.270655 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:53.270662 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:53.270787 (+ 125us) server_negotiation.cc:407] Connection header received
0810 14:45:53.271052 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:53.271056 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:53.271192 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:53.271348 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:45:53.271752 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:53.272508 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:53.273850 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:53.274278 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:53.274426 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:53.274930 (+ 504us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:45:53.275517 (+ 587us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:45:53.275689 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:53.275743 (+ 54us) server_negotiation.cc:299] Negotiation successful
0810 14:45:53.275825 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 14:45:53.283988 5759 raft_consensus.cc:1261] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43 [term 1 FOLLOWER]: Refusing update from remote peer 0b7f46fe1f6f46fc9a453321dd40b82f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:45:53.283991 5595 raft_consensus.cc:1261] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38 [term 1 FOLLOWER]: Refusing update from remote peer 0b7f46fe1f6f46fc9a453321dd40b82f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:45:53.284935 6013 consensus_queue.cc:1035] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [LEADER]: Connected to new peer: Peer: permanent_uuid: "19b17aad46894363bd2ba52127894c43" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41827 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:53.285282 6010 consensus_queue.cc:1035] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [LEADER]: Connected to new peer: Peer: permanent_uuid: "500cf995c6e64e9c84149212bec95c38" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46831 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:45:53.299263 6020 mvcc.cc:205] Tried to move back new op lower bound from 7316984435841282048 to 7316984435530506240. Current Snapshot: MvccSnapshot[applied={T|T < 7316984435841282048}]
I20260810 14:45:53.305444 6018 mvcc.cc:205] Tried to move back new op lower bound from 7316984435841282048 to 7316984435530506240. Current Snapshot: MvccSnapshot[applied={T|T < 7316984435841282048 or (T in {7316984435841282048})}]
I20260810 14:45:53.316586 5421 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:45:53.316879 5421 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:45:53.320210 5421 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7293ae109ebb4deca41ab1466983fdd5: Sending DeleteTablet for 3 replicas of tablet d3e6a100a7d4422792804964cbc7f80a
I20260810 14:45:53.321382 5572 tablet_service.cc:1514] Processing DeleteTablet for tablet d3e6a100a7d4422792804964cbc7f80a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:53 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:52986
I20260810 14:45:53.321382 5733 tablet_service.cc:1514] Processing DeleteTablet for tablet d3e6a100a7d4422792804964cbc7f80a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:53 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:38356
I20260810 14:45:53.321801 5895 tablet_service.cc:1514] Processing DeleteTablet for tablet d3e6a100a7d4422792804964cbc7f80a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:45:53 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:33050
I20260810 14:45:53.323892 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 5497
I20260810 14:45:53.337240 6034 tablet_replica.cc:330] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f: stopping tablet replica
I20260810 14:45:53.337741 6034 raft_consensus.cc:2229] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:45:53.338083 6033 tablet_replica.cc:330] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43: stopping tablet replica
I20260810 14:45:53.338512 6033 raft_consensus.cc:2229] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:45:53.339041 6033 raft_consensus.cc:2258] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:53.339869 6032 tablet_replica.cc:330] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38: stopping tablet replica
I20260810 14:45:53.340135 6034 raft_consensus.cc:2258] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:53.340248 6032 raft_consensus.cc:2229] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:45:53.340741 6032 raft_consensus.cc:2258] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:45:53.341952 6033 ts_tablet_manager.cc:1905] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:53.342126 6034 ts_tablet_manager.cc:1905] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:53.347071 6032 ts_tablet_manager.cc:1905] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:45:53.349761 6034 ts_tablet_manager.cc:1918] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:45:53.349946 6034 log.cc:1196] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/d3e6a100a7d4422792804964cbc7f80a
I20260810 14:45:53.350569 6034 ts_tablet_manager.cc:1939] T d3e6a100a7d4422792804964cbc7f80a P 0b7f46fe1f6f46fc9a453321dd40b82f: Deleting consensus metadata
I20260810 14:45:53.352535 5405 catalog_manager.cc:4841] TS 0b7f46fe1f6f46fc9a453321dd40b82f (127.24.215.131:32989): tablet d3e6a100a7d4422792804964cbc7f80a (table test-table [id=d177b0ebb6a1429d8a85a75754e002c2]) successfully deleted
I20260810 14:45:53.353248 6033 ts_tablet_manager.cc:1918] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:45:53.353420 6033 log.cc:1196] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/d3e6a100a7d4422792804964cbc7f80a
I20260810 14:45:53.353945 6033 ts_tablet_manager.cc:1939] T d3e6a100a7d4422792804964cbc7f80a P 19b17aad46894363bd2ba52127894c43: Deleting consensus metadata
Aug 10 14:45:53 dist-test-slave-vkjp krb5kdc[5369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:45:53.354871 6032 ts_tablet_manager.cc:1918] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:45:53.355163 6032 log.cc:1196] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/d3e6a100a7d4422792804964cbc7f80a
I20260810 14:45:53.355865 5404 catalog_manager.cc:4841] TS 19b17aad46894363bd2ba52127894c43 (127.24.215.130:41827): tablet d3e6a100a7d4422792804964cbc7f80a (table test-table [id=d177b0ebb6a1429d8a85a75754e002c2]) successfully deleted
I20260810 14:45:53.355965 6032 ts_tablet_manager.cc:1939] T d3e6a100a7d4422792804964cbc7f80a P 500cf995c6e64e9c84149212bec95c38: Deleting consensus metadata
I20260810 14:45:53.357748 5406 catalog_manager.cc:4841] TS 500cf995c6e64e9c84149212bec95c38 (127.24.215.129:46831): tablet d3e6a100a7d4422792804964cbc7f80a (table test-table [id=d177b0ebb6a1429d8a85a75754e002c2]) successfully deleted
I20260810 14:45:53.362742 5996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:53.325849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52994 (local address 127.24.215.129:46831)
0810 14:45:53.326072 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:53.326081 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:53.337449 (+ 11368us) server_negotiation.cc:407] Connection header received
0810 14:45:53.337653 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:53.337659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:53.337749 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:53.337864 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:53.342113 (+ 4249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:53.343267 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:53.345262 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:53.345948 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:53.346066 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:53.355638 (+ 9572us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:53.355653 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:53.355672 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:53.355725 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:53.359712 (+ 3987us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:53.360402 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:53.360412 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:53.360416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:53.360577 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:53.360944 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:53.360950 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:53.360953 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:53.361428 (+ 475us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:53.361744 (+ 316us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:53.362096 (+ 352us) server_negotiation.cc:299] Negotiation successful
0810 14:45:53.362317 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 14:45:53.363502 5649 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:53.454183 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 5497
I20260810 14:45:53.475994 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 5661
Aug 10 14:45:53 dist-test-slave-vkjp krb5kdc[5369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:45:53.493187 5998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:53.476830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38368 (local address 127.24.215.130:41827)
0810 14:45:53.477005 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:53.477010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:53.477064 (+ 54us) server_negotiation.cc:407] Connection header received
0810 14:45:53.477270 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:53.477274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:53.477338 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:53.477415 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:53.479121 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:53.479829 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:53.481671 (+ 1842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:53.482050 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:53.482139 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:53.487473 (+ 5334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:53.487490 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:53.487507 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:53.487545 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:53.490344 (+ 2799us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:53.491192 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:53.491199 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:53.491202 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:53.491335 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:53.491756 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:53.491760 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:53.491761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:53.492135 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:53.492342 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:53.492725 (+ 383us) server_negotiation.cc:299] Negotiation successful
0810 14:45:53.492884 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:45:53.493781 5809 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:53.644150 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 5661
I20260810 14:45:53.666404 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 5820
Aug 10 14:45:53 dist-test-slave-vkjp krb5kdc[5369](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:45:53.684082 5999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:45:53.667389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33076 (local address 127.24.215.131:32989)
0810 14:45:53.667601 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:45:53.667608 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:45:53.667636 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:45:53.667877 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:45:53.667884 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:45:53.667963 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:45:53.668119 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:45:53.669721 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:53.670617 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:53.672213 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:45:53.672758 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:45:53.672928 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:45:53.677607 (+ 4679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:45:53.677625 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:45:53.677648 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:45:53.677691 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:45:53.680971 (+ 3280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:53.681711 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:53.681718 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:53.681721 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:53.681873 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:45:53.682279 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:45:53.682283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:45:53.682285 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:45:53.682663 (+ 378us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:45:53.682994 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0810 14:45:53.683458 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 14:45:53.683697 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 14:45:53.684755 5972 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:53.809100 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 5820
I20260810 14:45:53.831528 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 5385
I20260810 14:45:53.832705 5458 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:45:53.943423 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 5385
2026-08-10T14:45:53Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4754 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:45:54 dist-test-slave-vkjp krb5kdc[6042](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:45:54 dist-test-slave-vkjp krb5kdc[6042](info): set up 2 sockets
Aug 10 14:45:54 dist-test-slave-vkjp krb5kdc[6042](info): commencing operation
krb5kdc: starting...
W20260810 14:45:55.145185 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:45:55 dist-test-slave-vkjp krb5kdc[6042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:45:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:45:55Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.215.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.215.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.215.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/rangeradmin_127.24.215.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.215.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/rangeradmin_127.24.215.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.215.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.215.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.215.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/rangerlookup_127.24.215.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.215.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/rangerlookup_127.24.215.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.215.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.215.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.215.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/HTTP_127.24.215.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.215.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/HTTP_127.24.215.148@KRBTEST.COM.keytab.
I20260810 14:45:55.534538 25438 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskvAcnHb/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/postgres -l logfile start
2026-08-10 14:45:58.182 UTC [6083] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 14:45:58.182 UTC [6083] LOG: listening on IPv4 address "127.24.215.148", port 51593
2026-08-10 14:45:58.186 UTC [6083] LOG: listening on Unix socket "/tmp/.s.PGSQL.51593"
2026-08-10 14:45:58.208 UTC [6086] LOG: database system was shut down at 2026-08-10 14:45:56 UTC
2026-08-10 14:45:58.212 UTC [6083] LOG: database system is ready to accept connections
I20260810 14:45:59.283052 25438 mini_postgres.cc:96] Postgres bound to 51593
2026-08-10 14:45:59.306 UTC [6094] FATAL: database "slave" does not exist
127.24.215.148:51593 - accepting connections
I20260810 14:45:59.307977 25438 mini_ranger.cc:161] Starting Ranger...
I20260810 14:45:59.340129 25438 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 14:45:59.857873 25438 mini_ranger.cc:87] Created ranger Postgres database
I20260810 14:45:59.858049 25438 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-admin
2026-08-10 14:46:00,200 [I] DB FLAVOR :POSTGRES
2026-08-10 14:46:00,200 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 14:46:00,200 [I] Checking connection..
2026-08-10 14:46:00,201 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 14:46:00,482 [I] Checking connection passed.
2026-08-10 14:46:00,482 [I] --------- Verifying version history table ---------
2026-08-10 14:46:00,483 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:46:00,758 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 14:46:00,759 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:46:01,032 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 14:46:01,033 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 14:46:01,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 14:46:01.272 UTC [6175] WARNING: there is no transaction in progress
2026-08-10 14:46:01,279 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 14:46:01,280 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:46:01,547 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 14:46:01,548 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 14:46:01,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 14:46:01,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 14:46:02,067 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vkjp.c.gcp-upstream.internal','N') ;"
2026-08-10 14:46:02,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 14:46:02,546 [I] Table x_portal_user does not exist in database ranger
2026-08-10 14:46:02,547 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 14:46:02,802 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 14:46:02,802 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 14:46:03,050 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 14:46:03,299 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 14:46:03,300 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 14:46:04.317 UTC [6305] WARNING: there is no transaction in progress
2026-08-10 14:46:04.333 UTC [6305] WARNING: there is no transaction in progress
2026-08-10 14:46:04.351 UTC [6305] WARNING: there is no transaction in progress
2026-08-10 14:46:04.368 UTC [6305] WARNING: there is no transaction in progress
2026-08-10 14:46:04.385 UTC [6305] WARNING: there is no transaction in progress
2026-08-10 14:46:04.403 UTC [6305] WARNING: there is no transaction in progress
2026-08-10 14:46:04.470 UTC [6305] WARNING: there is no transaction in progress
2026-08-10 14:46:04.478 UTC [6305] WARNING: there is no transaction in progress
2026-08-10 14:46:04.493 UTC [6305] WARNING: there is no transaction in progress
2026-08-10 14:46:04.504 UTC [6305] WARNING: there is no transaction in progress
2026-08-10 14:46:04.515 UTC [6305] WARNING: there is no transaction in progress
2026-08-10 14:46:04.529 UTC [6305] WARNING: there is no transaction in progress
2026-08-10 14:46:04.539 UTC [6305] WARNING: there is no transaction in progress
2026-08-10 14:46:04.546 UTC [6305] WARNING: there is no transaction in progress
2026-08-10 14:46:04.553 UTC [6305] WARNING: there is no transaction in progress
2026-08-10 14:46:04.560 UTC [6305] WARNING: there is no transaction in progress
2026-08-10 14:46:05,197 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 14:46:05,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 14:46:05,454 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 14:46:05,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 14:46:05,728 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 14:46:05,728 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 14:46:05,991 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 14:46:05,992 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 14:46:06,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 14:46:06,501 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vkjp.c.gcp-upstream.internal';"
2026-08-10 14:46:06,733 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 14:46:06,734 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 14:46:07,000 [I] DB_PATCHES have already been applied
I20260810 14:46:07.006757 25438 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-admin:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskvAcnHb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 14:46:07.006875 25438 mini_ranger.cc:193] Using host: 127.24.215.148
I20260810 14:46:07.027207 25438 mini_ranger.cc:227] Ranger admin URL: http://127.24.215.148:56835
Aug 10, 2026 2:46:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 2:46:07 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 2:46:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 2:46:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 2:46:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 2:46:08 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 2:46:08 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 14:46:08.250751 25438 mini_ranger.cc:160] Time spent starting Ranger: real 8.943s user 0.006s sys 0.052s
Aug 10, 2026 2:46:28 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 2:46:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 10, 2026 2:46:28 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 2:46:28 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 2:46:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 2:46:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 2:46:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 2:46:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 2:46:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 2:46:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 2:46:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 2:46:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 2:46:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 2:46:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 2:46:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 2:46:28 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 2:46:28 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 2:46:29 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 14:46:30.336513 25438 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.215.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.215.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.215.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/rangerkms_127.24.215.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.215.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/rangerkms_127.24.215.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.215.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.215.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.215.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/HTTP_127.24.215.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.215.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/HTTP_127.24.215.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 14:46:30 dist-test-slave-vkjp krb5kdc[6042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373190, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 14:46:30.544112 25438 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 14:46:30.571583 25438 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 14:46:31.406723 25438 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 14:46:31.406868 25438 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-kms
I20260810 14:46:31.407038 25438 mini_ranger_kms.cc:227] Using postgres at 127.24.215.148:51593
2026-08-10 14:46:32,201 [I] DB FLAVOR :POSTGRES
2026-08-10 14:46:32,201 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 14:46:32,201 [I] Checking connection
2026-08-10 14:46:32,201 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 14:46:32,464 [I] connection success
2026-08-10 14:46:32,464 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 14:46:32,465 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 14:46:32,465 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 14:46:32,782 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 14:46:32,782 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 14:46:32,782 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 14:46:32,782 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:51593/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 14:46:33,089 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 14:46:33.545598 25438 mini_ranger_kms.cc:310] Created kms service
I20260810 14:46:33.748438 25438 mini_ranger_kms.cc:326] Created kudu user
I20260810 14:46:33.794659 25438 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 14:46:33 dist-test-slave-vkjp krb5kdc[6042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373190, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:46:34.276742 25438 mini_ranger_kms.cc:384] Added ranger policy
I20260810 14:46:34.278184 25438 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-kms:/tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 14:46:34.278267 25438 mini_ranger_kms.cc:240] Using host: 127.24.215.148
I20260810 14:46:34.307384 25438 mini_ranger_kms.cc:276] Ranger KMS PID: 6526
I20260810 14:46:34.307606 25438 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.215.148:43471
Aug 10, 2026 2:46:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 2:46:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 2:46:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 2:46:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 2:46:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 2:46:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260810 14:46:36.926753 25438 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.383s user 0.156s sys 0.452s
I20260810 14:46:36.927212 25438 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 14:46:36.927462 25438 mini_ranger_kms.cc:401] 127.24.215.148:43471/kms/v1/keys
Aug 10 14:46:37 dist-test-slave-vkjp krb5kdc[6042](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786373197, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.215.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 10 14:46:37 dist-test-slave-vkjp krb5kdc[6042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373197, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.215.148@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
Aug 10 14:46:37 dist-test-slave-vkjp krb5kdc[6042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373197, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.215.148@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
Aug 10 14:46:40 dist-test-slave-vkjp krb5kdc[6042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:46:40.467623 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:37939
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:37939
--ranger_config_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.215.148:43471/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:46:40.927976 6592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:46:40.928347 6592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:46:40.928575 6592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:46:40.937378 6592 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:46:40.937549 6592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:46:40.937731 6592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:46:40.937858 6592 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:46:40.937945 6592 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:46:40.950783 6592 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.215.148:43471/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:37939
--ranger_config_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:37939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:46:40.952281 6592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:46:40.953641 6592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:46:40.968088 6598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:46:40.968299 6599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:46:40.968875 6603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:46:40.969509 6592 server_base.cc:1034] running on GCE node
I20260810 14:46:40.970431 6592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:46:40.971958 6592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:46:40.973183 6592 hybrid_clock.cc:648] HybridClock initialized: now 1786373200973138 us; error 40 us; skew 500 ppm
I20260810 14:46:40.973564 6592 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:46:40 dist-test-slave-vkjp krb5kdc[6042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373200, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:46:40.977043 6592 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:46:40.978649 6592 webserver.cc:460] Webserver started at http://127.24.215.190:38855/ using document root <none> and password file <none>
I20260810 14:46:40.979380 6592 fs_manager.cc:359] Metadata directory not provided
I20260810 14:46:40.979514 6592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:46:40.979799 6592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:46:40 dist-test-slave-vkjp krb5kdc[6042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373200, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:46:41.039454 6592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "a3d51a1eaf06443d9b1822655241ea32"
format_stamp: "Formatted at 2026-08-10 14:46:41 on dist-test-slave-vkjp"
server_key: "b9578001262c5da8247f70667bea0abf"
server_key_iv: "b0f94431d22b89a4faaf3a089e650317"
server_key_version: "kuduclusterkey@0"
I20260810 14:46:41.040382 6592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "a3d51a1eaf06443d9b1822655241ea32"
format_stamp: "Formatted at 2026-08-10 14:46:41 on dist-test-slave-vkjp"
server_key: "b9578001262c5da8247f70667bea0abf"
server_key_iv: "b0f94431d22b89a4faaf3a089e650317"
server_key_version: "kuduclusterkey@0"
I20260810 14:46:41.046283 6592 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 14:46:41.067819 6610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:46:41.069397 6592 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260810 14:46:41.069612 6592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "a3d51a1eaf06443d9b1822655241ea32"
format_stamp: "Formatted at 2026-08-10 14:46:41 on dist-test-slave-vkjp"
server_key: "b9578001262c5da8247f70667bea0abf"
server_key_iv: "b0f94431d22b89a4faaf3a089e650317"
server_key_version: "kuduclusterkey@0"
I20260810 14:46:41.069891 6592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:46:41.084226 6592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:46:41.088487 6592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:46:41.088858 6592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:46:41.114290 6592 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:37939
I20260810 14:46:41.114329 6669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:37939 every 8 connection(s)
I20260810 14:46:41.115826 6592 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:46:41.120152 6670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:46:41.123368 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 6592
I20260810 14:46:41.123584 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:46:41.124318 25438 external_mini_cluster.cc:1428] Setting key 937daa2b0c0677820e555a4c51c02095
I20260810 14:46:41.135862 6670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32: Bootstrap starting.
I20260810 14:46:41.139853 6670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32: Neither blocks nor log segments found. Creating new log.
Aug 10 14:46:41 dist-test-slave-vkjp krb5kdc[6042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:46:41.141096 6670 log.cc:824] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32: Log is configured to *not* fsync() on all Append() calls
I20260810 14:46:41.145505 6670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32: No bootstrap required, opened a new log
I20260810 14:46:41.151348 6673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:41.126525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34488 (local address 127.24.215.190:37939)
0810 14:46:41.127930 (+ 1405us) server_negotiation.cc:206] Beginning negotiation
0810 14:46:41.127947 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 14:46:41.128000 (+ 53us) server_negotiation.cc:407] Connection header received
0810 14:46:41.130613 (+ 2613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:46:41.130644 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:46:41.131219 (+ 575us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:46:41.131676 (+ 457us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:46:41.132813 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:41.135104 (+ 2291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:41.136393 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:41.137391 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:41.137620 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:41.141558 (+ 3938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:46:41.141570 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:46:41.141591 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:46:41.141679 (+ 88us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:46:41.146266 (+ 4587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:46:41.147036 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:46:41.147045 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:46:41.147053 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:46:41.147309 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:46:41.147690 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:46:41.147696 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:46:41.147701 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:46:41.148524 (+ 823us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:46:41.148872 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0810 14:46:41.149243 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 14:46:41.149693 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1099,"thread_start_us":778,"threads_started":1}
I20260810 14:46:41.156152 6670 raft_consensus.cc:348] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d51a1eaf06443d9b1822655241ea32" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 37939 } }
I20260810 14:46:41.156486 6670 raft_consensus.cc:374] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:46:41.156616 6670 raft_consensus.cc:729] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3d51a1eaf06443d9b1822655241ea32, State: Initialized, Role: FOLLOWER
I20260810 14:46:41.157267 6670 consensus_queue.cc:260] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d51a1eaf06443d9b1822655241ea32" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 37939 } }
I20260810 14:46:41.157531 6670 raft_consensus.cc:388] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:46:41.157739 6670 raft_consensus.cc:482] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:46:41.157969 6670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:46:41.160303 6670 raft_consensus.cc:504] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d51a1eaf06443d9b1822655241ea32" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 37939 } }
I20260810 14:46:41.160897 6670 leader_election.cc:302] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a3d51a1eaf06443d9b1822655241ea32; no voters:
I20260810 14:46:41.161499 6670 leader_election.cc:288] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:46:41.161708 6677 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:46:41.162607 6677 raft_consensus.cc:686] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [term 1 LEADER]: Becoming Leader. State: Replica: a3d51a1eaf06443d9b1822655241ea32, State: Running, Role: LEADER
I20260810 14:46:41.163266 6677 consensus_queue.cc:237] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d51a1eaf06443d9b1822655241ea32" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 37939 } }
I20260810 14:46:41.163846 6670 sys_catalog.cc:564] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:46:41.166365 6678 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a3d51a1eaf06443d9b1822655241ea32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d51a1eaf06443d9b1822655241ea32" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 37939 } } }
I20260810 14:46:41.166479 6679 sys_catalog.cc:455] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a3d51a1eaf06443d9b1822655241ea32. Latest consensus state: current_term: 1 leader_uuid: "a3d51a1eaf06443d9b1822655241ea32" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a3d51a1eaf06443d9b1822655241ea32" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 37939 } } }
I20260810 14:46:41.167150 6678 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [sys.catalog]: This master's current role is: LEADER
I20260810 14:46:41.167608 6679 sys_catalog.cc:458] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32 [sys.catalog]: This master's current role is: LEADER
I20260810 14:46:41.169390 6683 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:46:41.176182 6683 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:46:41.176995 6670 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 14:46:41.197717 6683 catalog_manager.cc:1347] Generated new cluster ID: 6752fb64c69249afb3f755be0bc1f2ad
I20260810 14:46:41.197885 6683 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:46:41.229876 6683 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:46:41.232436 6683 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:46:41.246645 6683 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32: Generated new TSK 0
I20260810 14:46:41.247840 6683 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:46:42.058736 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.215.148:43471/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:37939
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:46:42 dist-test-slave-vkjp krb5kdc[6042](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786373202, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 14:46:42.576435 6726 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:46:42.576900 6726 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:46:42.577167 6726 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:46:42.589900 6726 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:46:42.590039 6726 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:46:42.590273 6726 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:46:42.607012 6726 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.215.148:43471/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:37939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:46:42.609022 6726 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:46:42.611048 6726 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:46:42.630290 6737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:46:42.633388 6738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:46:42.633965 6741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:46:42.634727 6726 server_base.cc:1034] running on GCE node
I20260810 14:46:42.635459 6726 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:46:42.636579 6726 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:46:42.637988 6726 hybrid_clock.cc:648] HybridClock initialized: now 1786373202637886 us; error 84 us; skew 500 ppm
I20260810 14:46:42.638399 6726 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:46:42 dist-test-slave-vkjp krb5kdc[6042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373202, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:46:42.642118 6726 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:46:42.644312 6726 webserver.cc:460] Webserver started at http://127.24.215.129:37093/ using document root <none> and password file <none>
I20260810 14:46:42.645162 6726 fs_manager.cc:359] Metadata directory not provided
I20260810 14:46:42.645316 6726 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:46:42.645768 6726 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:46:42 dist-test-slave-vkjp krb5kdc[6042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373202, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:46:42.668145 6726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "360c48fd959143be942ce3a373a62dec"
format_stamp: "Formatted at 2026-08-10 14:46:42 on dist-test-slave-vkjp"
server_key: "923fbb00e3bc8f94b4fee01f20d13d42"
server_key_iv: "bf2cd47378b03ed81160512ed4f45c20"
server_key_version: "kuduclusterkey@0"
I20260810 14:46:42.669324 6726 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "360c48fd959143be942ce3a373a62dec"
format_stamp: "Formatted at 2026-08-10 14:46:42 on dist-test-slave-vkjp"
server_key: "923fbb00e3bc8f94b4fee01f20d13d42"
server_key_iv: "bf2cd47378b03ed81160512ed4f45c20"
server_key_version: "kuduclusterkey@0"
I20260810 14:46:42.686512 6726 fs_manager.cc:691] Time spent creating directory manager: real 0.016s user 0.007s sys 0.007s
I20260810 14:46:42.729347 6749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:46:42.731689 6726 fs_manager.cc:725] Time spent opening block manager: real 0.028s user 0.005s sys 0.020s
I20260810 14:46:42.731894 6726 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "360c48fd959143be942ce3a373a62dec"
format_stamp: "Formatted at 2026-08-10 14:46:42 on dist-test-slave-vkjp"
server_key: "923fbb00e3bc8f94b4fee01f20d13d42"
server_key_iv: "bf2cd47378b03ed81160512ed4f45c20"
server_key_version: "kuduclusterkey@0"
I20260810 14:46:42.732177 6726 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:46:42.770596 6726 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:46:42.777189 6726 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:46:42.777572 6726 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:46:42.778934 6726 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:46:42.780982 6726 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:46:42.781138 6726 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:46:42.781330 6726 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:46:42.781421 6726 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:46:42 dist-test-slave-vkjp krb5kdc[6042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373202, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:46:42.927696 6726 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:45211
I20260810 14:46:42.927762 6885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:45211 every 8 connection(s)
I20260810 14:46:42.929903 6726 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:46:42.932663 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 6726
I20260810 14:46:42.932896 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:46:42.933950 25438 external_mini_cluster.cc:1428] Setting key b815912ac996a5be9ed4ca350afb1768
Aug 10 14:46:42 dist-test-slave-vkjp krb5kdc[6042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373202, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:46:42.981217 6888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:42.936076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:59993 (local address 127.24.215.190:37939)
0810 14:46:42.952765 (+ 16689us) server_negotiation.cc:206] Beginning negotiation
0810 14:46:42.952770 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:46:42.957451 (+ 4681us) server_negotiation.cc:407] Connection header received
0810 14:46:42.960562 (+ 3111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:46:42.960568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:46:42.960656 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:46:42.960732 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:46:42.963330 (+ 2598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:42.964440 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:42.967517 (+ 3077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:42.968164 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:42.968257 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:42.972997 (+ 4740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:46:42.973004 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:46:42.973009 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:46:42.973049 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:46:42.976793 (+ 3744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:46:42.977740 (+ 947us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:46:42.977746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:46:42.977749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:46:42.977985 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:46:42.978709 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:46:42.978716 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:46:42.978719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:46:42.979112 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:46:42.979271 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 14:46:42.980340 (+ 1069us) server_negotiation.cc:299] Negotiation successful
0810 14:46:42.980716 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16562,"thread_start_us":16661,"threads_started":1}
I20260810 14:46:42.982414 6890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:42.936990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:37939 (local address 127.24.215.129:59993)
0810 14:46:42.957238 (+ 20248us) negotiation.cc:107] Waiting for socket to connect
0810 14:46:42.957300 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 14:46:42.960194 (+ 2894us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:46:42.961404 (+ 1210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:46:42.961416 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:46:42.962035 (+ 619us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:46:42.963118 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:46:42.963144 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:46:42.964658 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:46:42.964663 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:46:42.967349 (+ 2686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:46:42.967364 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:46:42.968618 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:46:42.968624 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:46:42.968915 (+ 291us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:42.969814 (+ 899us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:46:42.969846 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:46:42.972710 (+ 2864us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:46:42.976977 (+ 4267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:46:42.976986 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:46:42.977005 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:46:42.977599 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:46:42.978165 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:46:42.978169 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:46:42.978172 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:46:42.978537 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:46:42.979354 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:46:42.979363 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:46:42.979808 (+ 445us) client_negotiation.cc:769] Sending connection context
0810 14:46:42.980150 (+ 342us) client_negotiation.cc:240] Negotiation successful
0810 14:46:42.980763 (+ 613us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19803,"thread_start_us":20408,"threads_started":1}
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
I20260810 14:46:42.984439 6886 heartbeater.cc:347] Connected to a master server at 127.24.215.190:37939
I20260810 14:46:42.984768 6886 heartbeater.cc:464] Registering TS with master...
I20260810 14:46:42.985526 6886 heartbeater.cc:511] Master 127.24.215.190:37939 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
I20260810 14:46:42.988102 6628 ts_manager.cc:194] Registered new tserver with Master: 360c48fd959143be942ce3a373a62dec (127.24.215.129:45211)
I20260810 14:46:42.990316 6628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:59993
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:46:43.030741 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.215.148:43471/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:37939
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:46:43 dist-test-slave-vkjp krb5kdc[6042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373202, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
W20260810 14:46:43.614490 6893 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:46:43.614869 6893 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:46:43.615031 6893 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:46:43.627518 6893 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:46:43.627661 6893 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:46:43.627842 6893 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:46:43.644006 6893 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.215.148:43471/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:37939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:46:43.645788 6893 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:46:43.647560 6893 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:46:43.667325 6911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:46:43.667752 6913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:46:43.669225 6908 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:46:43.670252 6893 server_base.cc:1034] running on GCE node
I20260810 14:46:43.671029 6893 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:46:43.672358 6893 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:46:43.673661 6893 hybrid_clock.cc:648] HybridClock initialized: now 1786373203673535 us; error 59 us; skew 500 ppm
I20260810 14:46:43.674139 6893 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:46:43 dist-test-slave-vkjp krb5kdc[6042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373203, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:46:43.679118 6893 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:46:43.681479 6893 webserver.cc:460] Webserver started at http://127.24.215.130:38435/ using document root <none> and password file <none>
I20260810 14:46:43.682451 6893 fs_manager.cc:359] Metadata directory not provided
I20260810 14:46:43.682592 6893 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:46:43.682952 6893 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:46:43 dist-test-slave-vkjp krb5kdc[6042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373203, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:46:43.699180 6893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "11f8e58916674f99a9e54fc73d9a7ba5"
format_stamp: "Formatted at 2026-08-10 14:46:43 on dist-test-slave-vkjp"
server_key: "5611e9cdf01ccfa8ef059f18948f160f"
server_key_iv: "ea8b61fd0fd0ee25e2df5871d8a6e3de"
server_key_version: "kuduclusterkey@0"
I20260810 14:46:43.700101 6893 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "11f8e58916674f99a9e54fc73d9a7ba5"
format_stamp: "Formatted at 2026-08-10 14:46:43 on dist-test-slave-vkjp"
server_key: "5611e9cdf01ccfa8ef059f18948f160f"
server_key_iv: "ea8b61fd0fd0ee25e2df5871d8a6e3de"
server_key_version: "kuduclusterkey@0"
I20260810 14:46:43.706416 6893 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260810 14:46:43.726754 6922 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:46:43.728433 6893 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:46:43.728643 6893 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "11f8e58916674f99a9e54fc73d9a7ba5"
format_stamp: "Formatted at 2026-08-10 14:46:43 on dist-test-slave-vkjp"
server_key: "5611e9cdf01ccfa8ef059f18948f160f"
server_key_iv: "ea8b61fd0fd0ee25e2df5871d8a6e3de"
server_key_version: "kuduclusterkey@0"
I20260810 14:46:43.728893 6893 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:46:43.762382 6893 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:46:43.767694 6893 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:46:43.768015 6893 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:46:43.769168 6893 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:46:43.770944 6893 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:46:43.771023 6893 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:46:43.771188 6893 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:46:43.771288 6893 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:46:43.841086 6893 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:44235
I20260810 14:46:43.841113 7058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:44235 every 8 connection(s)
I20260810 14:46:43.842451 6893 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:46:43.844516 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 6893
I20260810 14:46:43.844697 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:46:43.845129 25438 external_mini_cluster.cc:1428] Setting key 7c3bc3e7da36e582c52fb532bea53c25
Aug 10 14:46:43 dist-test-slave-vkjp krb5kdc[6042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373203, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:46:43.873906 7061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:43.845722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:52289 (local address 127.24.215.190:37939)
0810 14:46:43.846453 (+ 731us) server_negotiation.cc:206] Beginning negotiation
0810 14:46:43.846490 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0810 14:46:43.848205 (+ 1715us) server_negotiation.cc:407] Connection header received
0810 14:46:43.850793 (+ 2588us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:46:43.850801 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:46:43.850942 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:46:43.851197 (+ 255us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:46:43.853941 (+ 2744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:43.855144 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:43.858356 (+ 3212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:43.859180 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:43.859338 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:43.864393 (+ 5055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:46:43.864404 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:46:43.864412 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:46:43.864461 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:46:43.869092 (+ 4631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:46:43.870075 (+ 983us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:46:43.870085 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:46:43.870091 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:46:43.870346 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:46:43.870956 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:46:43.870965 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:46:43.870970 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:46:43.871533 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:46:43.871753 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0810 14:46:43.872925 (+ 1172us) server_negotiation.cc:299] Negotiation successful
0810 14:46:43.873234 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":447,"threads_started":1}
I20260810 14:46:43.874982 7062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:43.846414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:37939 (local address 127.24.215.130:52289)
0810 14:46:43.847673 (+ 1259us) negotiation.cc:107] Waiting for socket to connect
0810 14:46:43.847730 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 14:46:43.850434 (+ 2704us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:46:43.851573 (+ 1139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:46:43.851588 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:46:43.852468 (+ 880us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:46:43.853699 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:46:43.853734 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:46:43.855356 (+ 1622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:46:43.855364 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:46:43.858101 (+ 2737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:46:43.858128 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:46:43.859503 (+ 1375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:46:43.859512 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:46:43.859834 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:43.860796 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:46:43.860826 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:46:43.864096 (+ 3270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:46:43.869278 (+ 5182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:46:43.869289 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:46:43.869310 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:46:43.869921 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:46:43.870499 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:46:43.870505 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:46:43.870510 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:46:43.870811 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:46:43.871757 (+ 946us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:46:43.871767 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:46:43.872321 (+ 554us) client_negotiation.cc:769] Sending connection context
0810 14:46:43.872738 (+ 417us) client_negotiation.cc:240] Negotiation successful
0810 14:46:43.873398 (+ 660us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":815,"spinlock_wait_cycles":50560,"thread_start_us":699,"threads_started":1}
I20260810 14:46:43.876889 7059 heartbeater.cc:347] Connected to a master server at 127.24.215.190:37939
I20260810 14:46:43.877224 7059 heartbeater.cc:464] Registering TS with master...
I20260810 14:46:43.878132 7059 heartbeater.cc:511] Master 127.24.215.190:37939 requested a full tablet report, sending...
I20260810 14:46:43.880281 6628 ts_manager.cc:194] Registered new tserver with Master: 11f8e58916674f99a9e54fc73d9a7ba5 (127.24.215.130:44235)
I20260810 14:46:43.881775 6628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:52289
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:46:43.917806 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.215.148:43471/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:37939
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:46:43.994771 6886 heartbeater.cc:503] Master 127.24.215.190:37939 was elected leader, sending a full tablet report...
W20260810 14:46:44.378319 7067 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:46:44.378619 7067 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:46:44.378808 7067 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:46:44.387228 7067 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:46:44.387421 7067 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:46:44.387598 7067 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:46:44.397871 7067 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.215.148:43471/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:37939
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:46:44.399286 7067 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:46:44.400664 7067 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:46:44.414624 7073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:46:44.414743 7076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:46:44.414737 7074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:46:44.415953 7067 server_base.cc:1034] running on GCE node
I20260810 14:46:44.416663 7067 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:46:44.417622 7067 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:46:44.418852 7067 hybrid_clock.cc:648] HybridClock initialized: now 1786373204418814 us; error 48 us; skew 500 ppm
I20260810 14:46:44.419263 7067 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:46:44 dist-test-slave-vkjp krb5kdc[6042](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373204, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:46:44.425336 7067 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:46:44.426891 7067 webserver.cc:460] Webserver started at http://127.24.215.131:44369/ using document root <none> and password file <none>
I20260810 14:46:44.427702 7067 fs_manager.cc:359] Metadata directory not provided
I20260810 14:46:44.427888 7067 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:46:44.428239 7067 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:46:44 dist-test-slave-vkjp krb5kdc[6042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373204, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:46:44.443032 7067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "7e9428b4b9b84fcd94ccd20c0fc57187"
format_stamp: "Formatted at 2026-08-10 14:46:44 on dist-test-slave-vkjp"
server_key: "9b808fbe72eae20c0c941085325baf14"
server_key_iv: "d6115fcf1c8e130a4567f364a436c119"
server_key_version: "kuduclusterkey@0"
I20260810 14:46:44.443949 7067 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "7e9428b4b9b84fcd94ccd20c0fc57187"
format_stamp: "Formatted at 2026-08-10 14:46:44 on dist-test-slave-vkjp"
server_key: "9b808fbe72eae20c0c941085325baf14"
server_key_iv: "d6115fcf1c8e130a4567f364a436c119"
server_key_version: "kuduclusterkey@0"
I20260810 14:46:44.449486 7067 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260810 14:46:44.463976 7087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:46:44.465647 7067 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:46:44.465832 7067 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "7e9428b4b9b84fcd94ccd20c0fc57187"
format_stamp: "Formatted at 2026-08-10 14:46:44 on dist-test-slave-vkjp"
server_key: "9b808fbe72eae20c0c941085325baf14"
server_key_iv: "d6115fcf1c8e130a4567f364a436c119"
server_key_version: "kuduclusterkey@0"
I20260810 14:46:44.466073 7067 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:46:44.477370 7067 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:46:44.481057 7067 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:46:44.481350 7067 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:46:44.482306 7067 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:46:44.483817 7067 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:46:44.483917 7067 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:46:44.484055 7067 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:46:44.484125 7067 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:46:44.523478 7067 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:38173
I20260810 14:46:44.523524 7217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:38173 every 8 connection(s)
I20260810 14:46:44.524884 7067 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:46:44.526149 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 7067
I20260810 14:46:44.526324 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:46:44.526787 25438 external_mini_cluster.cc:1428] Setting key b1aaa59458c0c82626be3aaf1871853e
Aug 10 14:46:44 dist-test-slave-vkjp krb5kdc[6042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373204, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:46:44.548027 7220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:44.528037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:59833 (local address 127.24.215.190:37939)
0810 14:46:44.528687 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0810 14:46:44.528693 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:46:44.529778 (+ 1085us) server_negotiation.cc:407] Connection header received
0810 14:46:44.532007 (+ 2229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:46:44.532011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:46:44.532075 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:46:44.532165 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:46:44.533909 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.534639 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.536434 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.536878 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.536971 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:44.541221 (+ 4250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:46:44.541227 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:46:44.541231 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:46:44.541267 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:46:44.545099 (+ 3832us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:46:44.545877 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:46:44.545882 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:46:44.545884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:46:44.546024 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:46:44.546441 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:46:44.546445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:46:44.546447 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:46:44.546721 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:46:44.546854 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 14:46:44.547581 (+ 727us) server_negotiation.cc:299] Negotiation successful
0810 14:46:44.547757 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":389,"threads_started":1}
I20260810 14:46:44.549029 7221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:44.528673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:37939 (local address 127.24.215.131:59833)
0810 14:46:44.529581 (+ 908us) negotiation.cc:107] Waiting for socket to connect
0810 14:46:44.529631 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 14:46:44.531746 (+ 2115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:46:44.532505 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:46:44.532521 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:46:44.532994 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:46:44.533740 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:46:44.533761 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.534814 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.534819 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:46:44.536296 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:46:44.536311 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.537094 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.537099 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:46:44.537377 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:44.538200 (+ 823us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:46:44.538225 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:46:44.540917 (+ 2692us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:46:44.545240 (+ 4323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:46:44.545248 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:46:44.545262 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:46:44.545713 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:46:44.546131 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:46:44.546135 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:46:44.546137 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:46:44.546331 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:46:44.546835 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:46:44.546841 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:46:44.547237 (+ 396us) client_negotiation.cc:769] Sending connection context
0810 14:46:44.547491 (+ 254us) client_negotiation.cc:240] Negotiation successful
0810 14:46:44.547934 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":607,"thread_start_us":390,"threads_started":1}
I20260810 14:46:44.550506 7218 heartbeater.cc:347] Connected to a master server at 127.24.215.190:37939
I20260810 14:46:44.550871 7218 heartbeater.cc:464] Registering TS with master...
I20260810 14:46:44.551836 7218 heartbeater.cc:511] Master 127.24.215.190:37939 requested a full tablet report, sending...
I20260810 14:46:44.553520 6628 ts_manager.cc:194] Registered new tserver with Master: 7e9428b4b9b84fcd94ccd20c0fc57187 (127.24.215.131:38173)
I20260810 14:46:44.554781 6628 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:59833
I20260810 14:46:44.555928 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:46:44.576124 7220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:44.561237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34492 (local address 127.24.215.190:37939)
0810 14:46:44.561404 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 14:46:44.561409 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:46:44.562166 (+ 757us) server_negotiation.cc:407] Connection header received
0810 14:46:44.562321 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:46:44.562325 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:46:44.562401 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:46:44.562487 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:46:44.564305 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.565118 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.566598 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.567238 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.567388 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:44.570043 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:46:44.570051 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:46:44.570057 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:46:44.570100 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:46:44.573150 (+ 3050us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:46:44.573987 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:46:44.573993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:46:44.573996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:46:44.574154 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:46:44.574583 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:46:44.574588 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:46:44.574590 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:46:44.574921 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:46:44.575185 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0810 14:46:44.575650 (+ 465us) server_negotiation.cc:299] Negotiation successful
0810 14:46:44.575844 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 14:46:44.580914 6628 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34492:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:46:44.588855 6628 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:46:44.618716 7232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:44.603255 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:44235 (local address 127.0.0.1:33918)
0810 14:46:44.604472 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0810 14:46:44.604515 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 14:46:44.604711 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:46:44.606667 (+ 1956us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:46:44.606678 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:46:44.606726 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:46:44.607486 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:46:44.607509 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.610338 (+ 2829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.610346 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:46:44.613717 (+ 3371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:46:44.613732 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.617239 (+ 3507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.617247 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:46:44.617572 (+ 325us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:44.617634 (+ 62us) client_negotiation.cc:769] Sending connection context
0810 14:46:44.617745 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 14:46:44.617944 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":806,"thread_start_us":549,"threads_started":1}
I20260810 14:46:44.620362 7233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:44.603501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33918 (local address 127.24.215.130:44235)
0810 14:46:44.604982 (+ 1481us) server_negotiation.cc:206] Beginning negotiation
0810 14:46:44.604991 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:46:44.605020 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:46:44.605149 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:46:44.605158 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:46:44.606093 (+ 935us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:46:44.606371 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:46:44.607824 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.610123 (+ 2299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.614198 (+ 4075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.616400 (+ 2202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.616630 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:44.619230 (+ 2600us) server_negotiation.cc:1035] Waiting for connection context
0810 14:46:44.619400 (+ 170us) server_negotiation.cc:299] Negotiation successful
0810 14:46:44.619717 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1307,"thread_start_us":1201,"threads_started":1}
I20260810 14:46:44.625910 6977 tablet_service.cc:1467] Processing CreateTablet for tablet 077d9fe036cb4cf993221f44eb33976b (DEFAULT_TABLE table=test-table [id=d5d32331d7df4f7eace4f75f86ab6573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:46:44.627667 6977 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077d9fe036cb4cf993221f44eb33976b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:46:44.630642 7234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:44.605450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:38173 (local address 127.0.0.1:42976)
0810 14:46:44.613436 (+ 7986us) negotiation.cc:107] Waiting for socket to connect
0810 14:46:44.613463 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 14:46:44.613656 (+ 193us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:46:44.622249 (+ 8593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:46:44.622258 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:46:44.622298 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:46:44.622899 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:46:44.622914 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.625045 (+ 2131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.625051 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:46:44.627576 (+ 2525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:46:44.627589 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.629755 (+ 2166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.629765 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:46:44.629964 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:44.630004 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 14:46:44.630102 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 14:46:44.630246 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7734,"thread_start_us":7453,"threads_started":1}
I20260810 14:46:44.631448 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:44.606051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42976 (local address 127.24.215.131:38173)
0810 14:46:44.621547 (+ 15496us) server_negotiation.cc:206] Beginning negotiation
0810 14:46:44.621559 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:46:44.621607 (+ 48us) server_negotiation.cc:407] Connection header received
0810 14:46:44.621713 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:46:44.621720 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:46:44.621993 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:46:44.622213 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:46:44.623146 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.624898 (+ 1752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.627737 (+ 2839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.629471 (+ 1734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.629720 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:44.629902 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 14:46:44.630883 (+ 981us) server_negotiation.cc:299] Negotiation successful
0810 14:46:44.631045 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15255,"thread_start_us":15807,"threads_started":1}
I20260810 14:46:44.631464 7237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:44.606484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51156 (local address 127.24.215.129:45211)
0810 14:46:44.623076 (+ 16592us) server_negotiation.cc:206] Beginning negotiation
0810 14:46:44.623110 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0810 14:46:44.623136 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:46:44.623223 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:46:44.623229 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:46:44.623407 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:46:44.623598 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:46:44.623980 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.625636 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.628049 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.629920 (+ 1871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.630212 (+ 292us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:44.630373 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 14:46:44.630557 (+ 184us) server_negotiation.cc:299] Negotiation successful
0810 14:46:44.630956 (+ 399us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16384,"thread_start_us":14407,"threads_started":1}
I20260810 14:46:44.630932 7232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:44.605937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:45211 (local address 127.0.0.1:51156)
0810 14:46:44.620827 (+ 14890us) negotiation.cc:107] Waiting for socket to connect
0810 14:46:44.620853 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:46:44.620968 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:46:44.623585 (+ 2617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:46:44.623590 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:46:44.623615 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:46:44.623841 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:46:44.623852 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.625880 (+ 2028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.625887 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:46:44.627842 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:46:44.627868 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.630073 (+ 2205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.630081 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:46:44.630319 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:44.630351 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 14:46:44.630458 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 14:46:44.630601 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14756,"thread_start_us":16650,"threads_started":1}
I20260810 14:46:44.635690 6803 tablet_service.cc:1467] Processing CreateTablet for tablet 077d9fe036cb4cf993221f44eb33976b (DEFAULT_TABLE table=test-table [id=d5d32331d7df4f7eace4f75f86ab6573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:46:44.635972 7138 tablet_service.cc:1467] Processing CreateTablet for tablet 077d9fe036cb4cf993221f44eb33976b (DEFAULT_TABLE table=test-table [id=d5d32331d7df4f7eace4f75f86ab6573]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:46:44.637045 6803 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077d9fe036cb4cf993221f44eb33976b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:46:44.637950 7138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 077d9fe036cb4cf993221f44eb33976b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:46:44.646695 7238 tablet_bootstrap.cc:492] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5: Bootstrap starting.
I20260810 14:46:44.651052 7239 tablet_bootstrap.cc:492] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec: Bootstrap starting.
I20260810 14:46:44.652383 7238 tablet_bootstrap.cc:654] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5: Neither blocks nor log segments found. Creating new log.
I20260810 14:46:44.653589 7238 log.cc:824] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5: Log is configured to *not* fsync() on all Append() calls
I20260810 14:46:44.655911 7239 tablet_bootstrap.cc:654] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec: Neither blocks nor log segments found. Creating new log.
I20260810 14:46:44.656831 7238 tablet_bootstrap.cc:492] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5: No bootstrap required, opened a new log
I20260810 14:46:44.657126 7239 log.cc:824] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec: Log is configured to *not* fsync() on all Append() calls
I20260810 14:46:44.657155 7238 ts_tablet_manager.cc:1397] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260810 14:46:44.658128 7241 tablet_bootstrap.cc:492] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187: Bootstrap starting.
I20260810 14:46:44.661336 7239 tablet_bootstrap.cc:492] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec: No bootstrap required, opened a new log
I20260810 14:46:44.661736 7239 ts_tablet_manager.cc:1397] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260810 14:46:44.662551 7241 tablet_bootstrap.cc:654] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187: Neither blocks nor log segments found. Creating new log.
I20260810 14:46:44.663961 7241 log.cc:824] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187: Log is configured to *not* fsync() on all Append() calls
I20260810 14:46:44.666879 7238 raft_consensus.cc:348] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f8e58916674f99a9e54fc73d9a7ba5" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44235 } } peers { permanent_uuid: "7e9428b4b9b84fcd94ccd20c0fc57187" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38173 } } peers { permanent_uuid: "360c48fd959143be942ce3a373a62dec" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45211 } }
I20260810 14:46:44.667304 7241 tablet_bootstrap.cc:492] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187: No bootstrap required, opened a new log
I20260810 14:46:44.667299 7238 raft_consensus.cc:374] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:46:44.667476 7238 raft_consensus.cc:729] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f8e58916674f99a9e54fc73d9a7ba5, State: Initialized, Role: FOLLOWER
I20260810 14:46:44.667585 7241 ts_tablet_manager.cc:1397] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.004s
I20260810 14:46:44.668162 7238 consensus_queue.cc:260] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f8e58916674f99a9e54fc73d9a7ba5" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44235 } } peers { permanent_uuid: "7e9428b4b9b84fcd94ccd20c0fc57187" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38173 } } peers { permanent_uuid: "360c48fd959143be942ce3a373a62dec" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45211 } }
I20260810 14:46:44.669447 7059 heartbeater.cc:503] Master 127.24.215.190:37939 was elected leader, sending a full tablet report...
I20260810 14:46:44.670166 7238 ts_tablet_manager.cc:1428] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5: Time spent starting tablet: real 0.013s user 0.011s sys 0.004s
I20260810 14:46:44.673480 7239 raft_consensus.cc:348] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f8e58916674f99a9e54fc73d9a7ba5" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44235 } } peers { permanent_uuid: "7e9428b4b9b84fcd94ccd20c0fc57187" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38173 } } peers { permanent_uuid: "360c48fd959143be942ce3a373a62dec" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45211 } }
I20260810 14:46:44.673952 7239 raft_consensus.cc:374] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:46:44.674196 7239 raft_consensus.cc:729] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 360c48fd959143be942ce3a373a62dec, State: Initialized, Role: FOLLOWER
I20260810 14:46:44.674958 7239 consensus_queue.cc:260] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f8e58916674f99a9e54fc73d9a7ba5" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44235 } } peers { permanent_uuid: "7e9428b4b9b84fcd94ccd20c0fc57187" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38173 } } peers { permanent_uuid: "360c48fd959143be942ce3a373a62dec" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45211 } }
I20260810 14:46:44.676975 7239 ts_tablet_manager.cc:1428] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260810 14:46:44.679243 7241 raft_consensus.cc:348] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f8e58916674f99a9e54fc73d9a7ba5" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44235 } } peers { permanent_uuid: "7e9428b4b9b84fcd94ccd20c0fc57187" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38173 } } peers { permanent_uuid: "360c48fd959143be942ce3a373a62dec" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45211 } }
I20260810 14:46:44.679705 7241 raft_consensus.cc:374] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:46:44.679884 7241 raft_consensus.cc:729] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e9428b4b9b84fcd94ccd20c0fc57187, State: Initialized, Role: FOLLOWER
I20260810 14:46:44.680491 7241 consensus_queue.cc:260] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f8e58916674f99a9e54fc73d9a7ba5" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44235 } } peers { permanent_uuid: "7e9428b4b9b84fcd94ccd20c0fc57187" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38173 } } peers { permanent_uuid: "360c48fd959143be942ce3a373a62dec" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45211 } }
I20260810 14:46:44.681521 7218 heartbeater.cc:503] Master 127.24.215.190:37939 was elected leader, sending a full tablet report...
I20260810 14:46:44.682116 7241 ts_tablet_manager.cc:1428] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187: Time spent starting tablet: real 0.014s user 0.006s sys 0.008s
W20260810 14:46:44.704703 6887 tablet.cc:2369] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:46:44.776810 7219 tablet.cc:2369] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:46:44.837067 7245 raft_consensus.cc:482] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:46:44.837481 7245 raft_consensus.cc:504] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f8e58916674f99a9e54fc73d9a7ba5" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44235 } } peers { permanent_uuid: "7e9428b4b9b84fcd94ccd20c0fc57187" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38173 } } peers { permanent_uuid: "360c48fd959143be942ce3a373a62dec" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45211 } }
I20260810 14:46:44.839567 7245 leader_election.cc:288] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e9428b4b9b84fcd94ccd20c0fc57187 (127.24.215.131:38173), 360c48fd959143be942ce3a373a62dec (127.24.215.129:45211)
W20260810 14:46:44.845196 7060 tablet.cc:2369] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:46:44.849593 7237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:44.840565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:39331 (local address 127.24.215.129:45211)
0810 14:46:44.840833 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 14:46:44.840838 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:46:44.841784 (+ 946us) server_negotiation.cc:407] Connection header received
0810 14:46:44.842099 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:46:44.842105 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:46:44.842177 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:46:44.842258 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:46:44.843507 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.844594 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.847487 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.848528 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.848740 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:44.848850 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 14:46:44.849157 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 14:46:44.849292 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260810 14:46:44.850188 7249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:44.840064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:45211 (local address 127.24.215.130:39331)
0810 14:46:44.841644 (+ 1580us) negotiation.cc:107] Waiting for socket to connect
0810 14:46:44.841666 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:46:44.841913 (+ 247us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:46:44.842437 (+ 524us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:46:44.842441 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:46:44.842491 (+ 50us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:46:44.843322 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:46:44.843336 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.844755 (+ 1419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.844761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:46:44.847257 (+ 2496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:46:44.847286 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.848780 (+ 1494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.848787 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:46:44.848981 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:44.849008 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 14:46:44.849125 (+ 117us) client_negotiation.cc:240] Negotiation successful
0810 14:46:44.849264 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1426,"thread_start_us":1540,"threads_started":1}
I20260810 14:46:44.850883 7236 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:44.840592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:54743 (local address 127.24.215.131:38173)
0810 14:46:44.840805 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 14:46:44.840810 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:46:44.841716 (+ 906us) server_negotiation.cc:407] Connection header received
0810 14:46:44.842031 (+ 315us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:46:44.842038 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:46:44.842127 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:46:44.842236 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:46:44.843265 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.844463 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.847948 (+ 3485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.849183 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.849370 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:44.849441 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 14:46:44.850471 (+ 1030us) server_negotiation.cc:299] Negotiation successful
0810 14:46:44.850617 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 14:46:44.851454 6828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077d9fe036cb4cf993221f44eb33976b" candidate_uuid: "11f8e58916674f99a9e54fc73d9a7ba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "360c48fd959143be942ce3a373a62dec" is_pre_election: true
I20260810 14:46:44.851754 7248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:44.839862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:38173 (local address 127.24.215.130:54743)
0810 14:46:44.841593 (+ 1731us) negotiation.cc:107] Waiting for socket to connect
0810 14:46:44.841615 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:46:44.841856 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:46:44.842258 (+ 402us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:46:44.842263 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:46:44.842291 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:46:44.843043 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:46:44.843057 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.844630 (+ 1573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.844635 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:46:44.847779 (+ 3144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:46:44.847794 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.850014 (+ 2220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.850024 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:46:44.850288 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:44.850322 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 14:46:44.851134 (+ 812us) client_negotiation.cc:240] Negotiation successful
0810 14:46:44.851282 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1529,"spinlock_wait_cycles":17920,"thread_start_us":1413,"threads_started":1}
I20260810 14:46:44.852022 6828 raft_consensus.cc:2454] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f8e58916674f99a9e54fc73d9a7ba5 in term 0.
I20260810 14:46:44.852731 7166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077d9fe036cb4cf993221f44eb33976b" candidate_uuid: "11f8e58916674f99a9e54fc73d9a7ba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e9428b4b9b84fcd94ccd20c0fc57187" is_pre_election: true
I20260810 14:46:44.853173 7166 raft_consensus.cc:2454] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 11f8e58916674f99a9e54fc73d9a7ba5 in term 0.
I20260810 14:46:44.853834 6926 leader_election.cc:302] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f8e58916674f99a9e54fc73d9a7ba5, 7e9428b4b9b84fcd94ccd20c0fc57187; no voters:
I20260810 14:46:44.854229 7245 raft_consensus.cc:2781] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:46:44.854417 7245 raft_consensus.cc:482] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:46:44.854552 7245 raft_consensus.cc:3037] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:46:44.857929 7245 raft_consensus.cc:504] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f8e58916674f99a9e54fc73d9a7ba5" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44235 } } peers { permanent_uuid: "7e9428b4b9b84fcd94ccd20c0fc57187" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38173 } } peers { permanent_uuid: "360c48fd959143be942ce3a373a62dec" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45211 } }
I20260810 14:46:44.858606 7245 leader_election.cc:288] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [CANDIDATE]: Term 1 election: Requested vote from peers 7e9428b4b9b84fcd94ccd20c0fc57187 (127.24.215.131:38173), 360c48fd959143be942ce3a373a62dec (127.24.215.129:45211)
I20260810 14:46:44.859061 7166 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077d9fe036cb4cf993221f44eb33976b" candidate_uuid: "11f8e58916674f99a9e54fc73d9a7ba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e9428b4b9b84fcd94ccd20c0fc57187"
I20260810 14:46:44.859157 6828 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "077d9fe036cb4cf993221f44eb33976b" candidate_uuid: "11f8e58916674f99a9e54fc73d9a7ba5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "360c48fd959143be942ce3a373a62dec"
I20260810 14:46:44.859347 7166 raft_consensus.cc:3037] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:46:44.859372 6828 raft_consensus.cc:3037] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:46:44.861516 6828 raft_consensus.cc:2454] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f8e58916674f99a9e54fc73d9a7ba5 in term 1.
I20260810 14:46:44.861732 7166 raft_consensus.cc:2454] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 11f8e58916674f99a9e54fc73d9a7ba5 in term 1.
I20260810 14:46:44.861943 6924 leader_election.cc:302] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 11f8e58916674f99a9e54fc73d9a7ba5, 360c48fd959143be942ce3a373a62dec; no voters:
I20260810 14:46:44.862326 7245 raft_consensus.cc:2781] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:46:44.862797 7245 raft_consensus.cc:686] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [term 1 LEADER]: Becoming Leader. State: Replica: 11f8e58916674f99a9e54fc73d9a7ba5, State: Running, Role: LEADER
I20260810 14:46:44.863355 7245 consensus_queue.cc:237] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f8e58916674f99a9e54fc73d9a7ba5" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44235 } } peers { permanent_uuid: "7e9428b4b9b84fcd94ccd20c0fc57187" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38173 } } peers { permanent_uuid: "360c48fd959143be942ce3a373a62dec" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45211 } }
I20260810 14:46:44.868009 6628 catalog_manager.cc:5495] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 11f8e58916674f99a9e54fc73d9a7ba5 (127.24.215.130). New cstate: current_term: 1 leader_uuid: "11f8e58916674f99a9e54fc73d9a7ba5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "11f8e58916674f99a9e54fc73d9a7ba5" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7e9428b4b9b84fcd94ccd20c0fc57187" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "360c48fd959143be942ce3a373a62dec" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45211 } health_report { overall_health: UNKNOWN } } }
I20260810 14:46:44.917444 7233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:44.911307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33934 (local address 127.24.215.130:44235)
0810 14:46:44.911518 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 14:46:44.911525 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:46:44.911583 (+ 58us) server_negotiation.cc:407] Connection header received
0810 14:46:44.911830 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:46:44.911834 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:46:44.911916 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:46:44.912073 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:46:44.912577 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.913382 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.914987 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.915538 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:44.915677 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:44.916207 (+ 530us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:46:44.916708 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:46:44.916867 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 14:46:44.916991 (+ 124us) server_negotiation.cc:299] Negotiation successful
0810 14:46:44.917072 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 14:46:44.926458 6828 raft_consensus.cc:1261] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec [term 1 FOLLOWER]: Refusing update from remote peer 11f8e58916674f99a9e54fc73d9a7ba5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:46:44.926483 7166 raft_consensus.cc:1261] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187 [term 1 FOLLOWER]: Refusing update from remote peer 11f8e58916674f99a9e54fc73d9a7ba5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:46:44.927575 7250 consensus_queue.cc:1035] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "360c48fd959143be942ce3a373a62dec" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 45211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:46:44.927991 7245 consensus_queue.cc:1035] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e9428b4b9b84fcd94ccd20c0fc57187" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:46:44.940727 7254 mvcc.cc:205] Tried to move back new op lower bound from 7316984647368814592 to 7316984647124205568. Current Snapshot: MvccSnapshot[applied={T|T < 7316984647368814592}]
I20260810 14:46:44.967865 6628 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:46:44.968181 6628 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34492:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:46:44.972353 6628 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a3d51a1eaf06443d9b1822655241ea32: Sending DeleteTablet for 3 replicas of tablet 077d9fe036cb4cf993221f44eb33976b
I20260810 14:46:44.973565 6977 tablet_service.cc:1514] Processing DeleteTablet for tablet 077d9fe036cb4cf993221f44eb33976b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:46:44 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:33918
I20260810 14:46:44.973996 7138 tablet_service.cc:1514] Processing DeleteTablet for tablet 077d9fe036cb4cf993221f44eb33976b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:46:44 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:42976
I20260810 14:46:44.974021 6803 tablet_service.cc:1514] Processing DeleteTablet for tablet 077d9fe036cb4cf993221f44eb33976b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:46:44 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:51156
I20260810 14:46:44.977694 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 6726
I20260810 14:46:44.985428 7268 tablet_replica.cc:330] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187: stopping tablet replica
I20260810 14:46:44.985924 7266 tablet_replica.cc:330] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5: stopping tablet replica
I20260810 14:46:44.986150 7268 raft_consensus.cc:2229] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:46:44.986343 7266 raft_consensus.cc:2229] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:46:44.986806 7268 raft_consensus.cc:2258] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:46:44.987039 7266 raft_consensus.cc:2258] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:46:44.987795 7267 tablet_replica.cc:330] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec: stopping tablet replica
I20260810 14:46:44.988198 7267 raft_consensus.cc:2229] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:46:44.988824 7267 raft_consensus.cc:2258] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:46:44.989616 7268 ts_tablet_manager.cc:1905] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:46:44.990974 7267 ts_tablet_manager.cc:1905] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:46:44.996833 7268 ts_tablet_manager.cc:1918] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:46:44.998479 7267 ts_tablet_manager.cc:1918] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:46:44.998766 7267 log.cc:1196] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/077d9fe036cb4cf993221f44eb33976b
I20260810 14:46:44.999272 7266 ts_tablet_manager.cc:1905] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:46:44.999642 7267 ts_tablet_manager.cc:1939] T 077d9fe036cb4cf993221f44eb33976b P 360c48fd959143be942ce3a373a62dec: Deleting consensus metadata
I20260810 14:46:45.002071 6611 catalog_manager.cc:4841] TS 360c48fd959143be942ce3a373a62dec (127.24.215.129:45211): tablet 077d9fe036cb4cf993221f44eb33976b (table test-table [id=d5d32331d7df4f7eace4f75f86ab6573]) successfully deleted
I20260810 14:46:45.002846 7268 log.cc:1196] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/077d9fe036cb4cf993221f44eb33976b
I20260810 14:46:45.003512 7268 ts_tablet_manager.cc:1939] T 077d9fe036cb4cf993221f44eb33976b P 7e9428b4b9b84fcd94ccd20c0fc57187: Deleting consensus metadata
I20260810 14:46:45.005760 6613 catalog_manager.cc:4841] TS 7e9428b4b9b84fcd94ccd20c0fc57187 (127.24.215.131:38173): tablet 077d9fe036cb4cf993221f44eb33976b (table test-table [id=d5d32331d7df4f7eace4f75f86ab6573]) successfully deleted
Aug 10 14:46:45 dist-test-slave-vkjp krb5kdc[6042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:46:45.007835 7266 ts_tablet_manager.cc:1918] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:46:45.008029 7266 log.cc:1196] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/077d9fe036cb4cf993221f44eb33976b
I20260810 14:46:45.008749 7266 ts_tablet_manager.cc:1939] T 077d9fe036cb4cf993221f44eb33976b P 11f8e58916674f99a9e54fc73d9a7ba5: Deleting consensus metadata
I20260810 14:46:45.011137 6614 catalog_manager.cc:4841] TS 11f8e58916674f99a9e54fc73d9a7ba5 (127.24.215.130:44235): tablet 077d9fe036cb4cf993221f44eb33976b (table test-table [id=d5d32331d7df4f7eace4f75f86ab6573]) successfully deleted
I20260810 14:46:45.015811 7237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:44.979998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51162 (local address 127.24.215.129:45211)
0810 14:46:44.980283 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0810 14:46:44.980295 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:46:44.983975 (+ 3680us) server_negotiation.cc:407] Connection header received
0810 14:46:44.984221 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:46:44.984232 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:46:44.984380 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:46:44.984532 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:46:44.997141 (+ 12609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:44.998357 (+ 1216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:45.000526 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:45.001241 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:45.001371 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:45.008452 (+ 7081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:46:45.008478 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:46:45.008500 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:46:45.008554 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:46:45.012648 (+ 4094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:46:45.013653 (+ 1005us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:46:45.013661 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:46:45.013663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:46:45.013838 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:46:45.014281 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:46:45.014285 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:46:45.014286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:46:45.014692 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:46:45.014926 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0810 14:46:45.015325 (+ 399us) server_negotiation.cc:299] Negotiation successful
0810 14:46:45.015513 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 14:46:45.016477 6878 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:46:45.172062 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 6726
I20260810 14:46:45.196302 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 6893
Aug 10 14:46:45 dist-test-slave-vkjp krb5kdc[6042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:46:45.214641 7233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:45.197173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33936 (local address 127.24.215.130:44235)
0810 14:46:45.197354 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0810 14:46:45.197359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:46:45.197419 (+ 60us) server_negotiation.cc:407] Connection header received
0810 14:46:45.197613 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:46:45.197617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:46:45.197717 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:46:45.197803 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:46:45.199603 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:45.200398 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:45.202333 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:45.202758 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:45.202848 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:45.208319 (+ 5471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:46:45.208338 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:46:45.208356 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:46:45.208391 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:46:45.211660 (+ 3269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:46:45.212423 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:46:45.212432 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:46:45.212434 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:46:45.212603 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:46:45.213013 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:46:45.213019 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:46:45.213022 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:46:45.213383 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:46:45.213723 (+ 340us) server_negotiation.cc:1035] Waiting for connection context
0810 14:46:45.214017 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 14:46:45.214249 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 14:46:45.215507 7053 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:46:45.332932 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 6893
I20260810 14:46:45.355705 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 7067
Aug 10 14:46:45 dist-test-slave-vkjp krb5kdc[6042](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373200, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:46:45.371563 7272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:46:45.356438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42990 (local address 127.24.215.131:38173)
0810 14:46:45.356901 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0810 14:46:45.356905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:46:45.356922 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:46:45.356981 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:46:45.356985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:46:45.357045 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:46:45.357161 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:46:45.358529 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:45.359367 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:45.360651 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:46:45.361091 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:46:45.361209 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:46:45.365838 (+ 4629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:46:45.365852 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:46:45.365867 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:46:45.365901 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:46:45.368683 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:46:45.369398 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:46:45.369404 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:46:45.369407 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:46:45.369538 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:46:45.369905 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:46:45.369921 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:46:45.369923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:46:45.370304 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:46:45.370550 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0810 14:46:45.370924 (+ 374us) server_negotiation.cc:299] Negotiation successful
0810 14:46:45.371171 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":259,"threads_started":1}
I20260810 14:46:45.372196 7212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:46:45.461485 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 7067
I20260810 14:46:45.483057 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 6592
I20260810 14:46:45.484409 6664 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:46:45.598467 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 6592
I20260810 14:46:45.629406 25438 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 14:46:45.974709 25438 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 14:46:45.975029 25438 mini_ranger.cc:66] Stopping Ranger...
I20260810 14:46:46.320297 25438 mini_ranger.cc:68] Stopped Ranger
2026-08-10 14:46:46.320 UTC [6083] LOG: received smart shutdown request
2026-08-10 14:46:46.324 UTC [6083] LOG: background worker "logical replication launcher" (PID 6092) exited with exit code 1
2026-08-10 14:46:46.324 UTC [6087] LOG: shutting down
2026-08-10 14:46:46.494 UTC [6083] LOG: database system is shut down
2026-08-10T14:46:46Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52683 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:46:46 dist-test-slave-vkjp krb5kdc[7284](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:46:46 dist-test-slave-vkjp krb5kdc[7284](info): set up 2 sockets
Aug 10 14:46:46 dist-test-slave-vkjp krb5kdc[7284](info): commencing operation
krb5kdc: starting...
W20260810 14:46:47.844462 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.001s sys 0.034s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:46:48 dist-test-slave-vkjp krb5kdc[7284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373208, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:46:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:46:48Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.24.215.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.24.215.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.24.215.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/rangeradmin_127.24.215.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.24.215.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/rangeradmin_127.24.215.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.24.215.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.24.215.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.24.215.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/rangerlookup_127.24.215.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.24.215.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/rangerlookup_127.24.215.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.215.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.24.215.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.215.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/HTTP_127.24.215.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.215.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/HTTP_127.24.215.148@KRBTEST.COM.keytab.
I20260810 14:46:48.230486 25438 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskvAcnHb/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/postgres -l logfile start
2026-08-10 14:46:50.720 UTC [7321] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-10 14:46:50.720 UTC [7321] LOG: listening on IPv4 address "127.24.215.148", port 36765
2026-08-10 14:46:50.723 UTC [7321] LOG: listening on Unix socket "/tmp/.s.PGSQL.36765"
2026-08-10 14:46:50.747 UTC [7324] LOG: database system was shut down at 2026-08-10 14:46:49 UTC
2026-08-10 14:46:50.751 UTC [7321] LOG: database system is ready to accept connections
I20260810 14:46:51.818776 25438 mini_postgres.cc:96] Postgres bound to 36765
2026-08-10 14:46:51.842 UTC [7333] FATAL: database "slave" does not exist
127.24.215.148:36765 - accepting connections
I20260810 14:46:51.843019 25438 mini_ranger.cc:161] Starting Ranger...
I20260810 14:46:51.873844 25438 mini_ranger.cc:84] Created miniranger Postgres user
I20260810 14:46:52.345311 25438 mini_ranger.cc:87] Created ranger Postgres database
I20260810 14:46:52.345412 25438 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-admin
2026-08-10 14:46:52,640 [I] DB FLAVOR :POSTGRES
2026-08-10 14:46:52,641 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 14:46:52,641 [I] Checking connection..
2026-08-10 14:46:52,641 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-10 14:46:52,913 [I] Checking connection passed.
2026-08-10 14:46:52,914 [I] --------- Verifying version history table ---------
2026-08-10 14:46:52,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:46:53,173 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 14:46:53,174 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:46:53,444 [I] Table x_db_version_h does not exist in database ranger
2026-08-10 14:46:53,444 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-10 14:46:53,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-10 14:46:53.703 UTC [7407] WARNING: there is no transaction in progress
2026-08-10 14:46:53,711 [I] create_dbversion_catalog.sql file imported successfully
2026-08-10 14:46:53,711 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-10 14:46:53,948 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-10 14:46:53,949 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 14:46:53,949 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-10 14:46:54,213 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-10 14:46:54,469 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vkjp.c.gcp-upstream.internal','N') ;"
2026-08-10 14:46:54,694 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 14:46:54,950 [I] Table x_portal_user does not exist in database ranger
2026-08-10 14:46:54,950 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 14:46:55,196 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-10 14:46:55,197 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 14:46:55,457 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 14:46:55,724 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-10 14:46:55,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-10 14:46:56.707 UTC [7535] WARNING: there is no transaction in progress
2026-08-10 14:46:56.723 UTC [7535] WARNING: there is no transaction in progress
2026-08-10 14:46:56.741 UTC [7535] WARNING: there is no transaction in progress
2026-08-10 14:46:56.756 UTC [7535] WARNING: there is no transaction in progress
2026-08-10 14:46:56.773 UTC [7535] WARNING: there is no transaction in progress
2026-08-10 14:46:56.794 UTC [7535] WARNING: there is no transaction in progress
2026-08-10 14:46:56.861 UTC [7535] WARNING: there is no transaction in progress
2026-08-10 14:46:56.868 UTC [7535] WARNING: there is no transaction in progress
2026-08-10 14:46:56.883 UTC [7535] WARNING: there is no transaction in progress
2026-08-10 14:46:56.894 UTC [7535] WARNING: there is no transaction in progress
2026-08-10 14:46:56.904 UTC [7535] WARNING: there is no transaction in progress
2026-08-10 14:46:56.919 UTC [7535] WARNING: there is no transaction in progress
2026-08-10 14:46:56.928 UTC [7535] WARNING: there is no transaction in progress
2026-08-10 14:46:56.934 UTC [7535] WARNING: there is no transaction in progress
2026-08-10 14:46:56.941 UTC [7535] WARNING: there is no transaction in progress
2026-08-10 14:46:56.948 UTC [7535] WARNING: there is no transaction in progress
2026-08-10 14:46:57,579 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-10 14:46:57,580 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-10 14:46:57,842 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-10 14:46:57,843 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-10 14:46:58,107 [I] Table x_portal_user already exists in database 'ranger'
2026-08-10 14:46:58,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-10 14:46:58,341 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-10 14:46:58,341 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-10 14:46:58,606 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-10 14:46:58,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vkjp.c.gcp-upstream.internal';"
2026-08-10 14:46:59,137 [I] CORE_DB_SCHEMA import status has been updated
2026-08-10 14:46:59,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-10 14:46:59,389 [I] DB_PATCHES have already been applied
I20260810 14:46:59.395390 25438 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-admin:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskvAcnHb/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260810 14:46:59.395486 25438 mini_ranger.cc:193] Using host: 127.24.215.148
I20260810 14:46:59.414788 25438 mini_ranger.cc:227] Ranger admin URL: http://127.24.215.148:43819
Aug 10, 2026 2:46:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 2:46:59 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 2:46:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-admin/ews/webapp
Aug 10, 2026 2:46:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 10, 2026 2:46:59 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 10, 2026 2:47:00 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-admin/ews/webapp].
Aug 10, 2026 2:47:00 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260810 14:47:00.659729 25438 mini_ranger.cc:160] Time spent starting Ranger: real 8.817s user 0.002s sys 0.052s
Aug 10, 2026 2:47:21 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 10, 2026 2:47:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 10, 2026 2:47:22 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 10, 2026 2:47:22 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 10, 2026 2:47:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 10, 2026 2:47:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 10, 2026 2:47:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 10, 2026 2:47:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 10, 2026 2:47:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 10, 2026 2:47:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 10, 2026 2:47:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 10, 2026 2:47:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 10, 2026 2:47:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 10, 2026 2:47:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 10, 2026 2:47:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 10, 2026 2:47:22 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 10, 2026 2:47:22 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 10, 2026 2:47:22 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260810 14:47:23.483798 25438 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.24.215.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.24.215.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.24.215.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/rangerkms_127.24.215.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.24.215.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/rangerkms_127.24.215.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.24.215.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.24.215.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.24.215.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/HTTP_127.24.215.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.24.215.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/HTTP_127.24.215.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 10 14:47:23 dist-test-slave-vkjp krb5kdc[7284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373243, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260810 14:47:23.687853 25438 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260810 14:47:23.711621 25438 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260810 14:47:24.536923 25438 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260810 14:47:24.537037 25438 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-kms
I20260810 14:47:24.537097 25438 mini_ranger_kms.cc:227] Using postgres at 127.24.215.148:36765
2026-08-10 14:47:24,902 [I] DB FLAVOR :POSTGRES
2026-08-10 14:47:24,902 [I] --------- Verifying Ranger DB connection ---------
2026-08-10 14:47:24,903 [I] Checking connection
2026-08-10 14:47:24,903 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-10 14:47:25,177 [I] connection success
2026-08-10 14:47:25,177 [I] --------- Verifying Ranger DB tables ---------
2026-08-10 14:47:25,177 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-10 14:47:25,177 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-10 14:47:25,432 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-10 14:47:25,433 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-10 14:47:25,433 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-10 14:47:25,433 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.24.215.148:36765/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-10 14:47:25,706 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260810 14:47:26.167160 25438 mini_ranger_kms.cc:310] Created kms service
I20260810 14:47:26.362188 25438 mini_ranger_kms.cc:326] Created kudu user
I20260810 14:47:26.410318 25438 mini_ranger_kms.cc:343] Created rangerkms user
Aug 10 14:47:26 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373243, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:47:26.869438 25438 mini_ranger_kms.cc:384] Added ranger policy
I20260810 14:47:26.869817 25438 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-kms:/tmp/dist-test-taskvAcnHb/build/asan/bin/postgresql.jar:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskvAcnHb/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260810 14:47:26.869889 25438 mini_ranger_kms.cc:240] Using host: 127.24.215.148
I20260810 14:47:26.889173 25438 mini_ranger_kms.cc:276] Ranger KMS PID: 7747
I20260810 14:47:26.889312 25438 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.24.215.148:56975
Aug 10, 2026 2:47:27 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 10, 2026 2:47:27 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 10, 2026 2:47:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 10, 2026 2:47:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 10, 2026 2:47:28 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-kms/ews/webapp].
Aug 10, 2026 2:47:28 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 10 14:47:29 dist-test-slave-vkjp krb5kdc[7284](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786373249, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.215.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:29.366418 25438 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.679s user 0.054s sys 0.331s
I20260810 14:47:29.366663 25438 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260810 14:47:29.366860 25438 mini_ranger_kms.cc:401] 127.24.215.148:56975/kms/v1/keys
Aug 10 14:47:29 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373249, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.215.148@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
Aug 10 14:47:30 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373249, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.24.215.148@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
Aug 10 14:47:32 dist-test-slave-vkjp krb5kdc[7284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:47:32.494840 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:33629
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:43531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:33629
--ranger_config_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.215.148:56975/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:47:32.830425 7809 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:32.830722 7809 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:32.830906 7809 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:32.838195 7809 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:47:32.838301 7809 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:32.838424 7809 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:32.838526 7809 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:47:32.838601 7809 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:47:32.848676 7809 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:43531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/wal
--ranger_kms_url=127.24.215.148:56975/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:33629
--ranger_config_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:33629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:32.849987 7809 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:32.851336 7809 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:32.863545 7819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:32.863543 7816 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:32.863543 7817 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:32.864269 7809 server_base.cc:1034] running on GCE node
I20260810 14:47:32.865070 7809 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:32.866587 7809 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:32.867811 7809 hybrid_clock.cc:648] HybridClock initialized: now 1786373252867763 us; error 54 us; skew 500 ppm
I20260810 14:47:32.868178 7809 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:32 dist-test-slave-vkjp krb5kdc[7284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373252, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:32.875075 7809 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:47:32.876645 7809 webserver.cc:460] Webserver started at http://127.24.215.190:37401/ using document root <none> and password file <none>
I20260810 14:47:32.877271 7809 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:32.877367 7809 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:32.877626 7809 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:47:32 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373252, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:47:32.946511 7809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "9f12f4550b7240a8ae4a9c168395f4c5"
format_stamp: "Formatted at 2026-08-10 14:47:32 on dist-test-slave-vkjp"
server_key: "fd513bbbb730037f87add7c8aa723b5c"
server_key_iv: "340a7843ce1fa86d0dfe71e10da6dafa"
server_key_version: "kuduclusterkey@0"
I20260810 14:47:32.947459 7809 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "9f12f4550b7240a8ae4a9c168395f4c5"
format_stamp: "Formatted at 2026-08-10 14:47:32 on dist-test-slave-vkjp"
server_key: "fd513bbbb730037f87add7c8aa723b5c"
server_key_iv: "340a7843ce1fa86d0dfe71e10da6dafa"
server_key_version: "kuduclusterkey@0"
I20260810 14:47:32.953028 7809 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:47:32.973304 7829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:32.975221 7809 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:47:32.975461 7809 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "9f12f4550b7240a8ae4a9c168395f4c5"
format_stamp: "Formatted at 2026-08-10 14:47:32 on dist-test-slave-vkjp"
server_key: "fd513bbbb730037f87add7c8aa723b5c"
server_key_iv: "340a7843ce1fa86d0dfe71e10da6dafa"
server_key_version: "kuduclusterkey@0"
I20260810 14:47:32.975755 7809 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:32.992012 7809 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:32.997334 7809 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:32.997694 7809 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:33.018201 7809 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:33629
I20260810 14:47:33.018188 7892 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:33629 every 8 connection(s)
I20260810 14:47:33.019974 7809 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:47:33.024035 7894 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:47:33.024257 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 7809
I20260810 14:47:33.024492 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:47:33.025364 25438 external_mini_cluster.cc:1428] Setting key d77b11919d1a2955ad87fde280581176
I20260810 14:47:33.041396 7894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5: Bootstrap starting.
I20260810 14:47:33.045421 7894 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5: Neither blocks nor log segments found. Creating new log.
I20260810 14:47:33.046638 7894 log.cc:824] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5: Log is configured to *not* fsync() on all Append() calls
I20260810 14:47:33.053004 7894 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5: No bootstrap required, opened a new log
Aug 10 14:47:33 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:47:33.063930 7894 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f12f4550b7240a8ae4a9c168395f4c5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33629 } }
I20260810 14:47:33.064296 7897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:33.027633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59204 (local address 127.24.215.190:33629)
0810 14:47:33.039602 (+ 11969us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:33.039615 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:33.039659 (+ 44us) server_negotiation.cc:407] Connection header received
0810 14:47:33.041367 (+ 1708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:33.041396 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:33.041935 (+ 539us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:33.042356 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:33.044254 (+ 1898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:33.046156 (+ 1902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:33.048499 (+ 2343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:33.049521 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:33.049719 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:33.054822 (+ 5103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:33.054833 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:33.054854 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:33.054902 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:33.059311 (+ 4409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:33.060178 (+ 867us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:33.060188 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:33.060200 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:33.060399 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:33.060850 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:33.060855 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:33.060859 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:33.061668 (+ 809us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:33.061997 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:33.062507 (+ 510us) server_negotiation.cc:299] Negotiation successful
0810 14:47:33.062948 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11752,"thread_start_us":11541,"threads_started":1}
I20260810 14:47:33.064406 7894 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:47:33.064508 7894 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f12f4550b7240a8ae4a9c168395f4c5, State: Initialized, Role: FOLLOWER
I20260810 14:47:33.065220 7894 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f12f4550b7240a8ae4a9c168395f4c5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33629 } }
I20260810 14:47:33.065493 7894 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:47:33.065866 7894 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:47:33.066093 7894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:47:33.068858 7894 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f12f4550b7240a8ae4a9c168395f4c5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33629 } }
I20260810 14:47:33.069358 7894 leader_election.cc:302] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f12f4550b7240a8ae4a9c168395f4c5; no voters:
I20260810 14:47:33.070031 7894 leader_election.cc:288] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:47:33.070130 7900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:47:33.071468 7900 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 1 LEADER]: Becoming Leader. State: Replica: 9f12f4550b7240a8ae4a9c168395f4c5, State: Running, Role: LEADER
I20260810 14:47:33.072110 7900 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f12f4550b7240a8ae4a9c168395f4c5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33629 } }
I20260810 14:47:33.072528 7894 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:47:33.076936 7901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9f12f4550b7240a8ae4a9c168395f4c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f12f4550b7240a8ae4a9c168395f4c5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33629 } } }
I20260810 14:47:33.077011 7902 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f12f4550b7240a8ae4a9c168395f4c5. Latest consensus state: current_term: 1 leader_uuid: "9f12f4550b7240a8ae4a9c168395f4c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f12f4550b7240a8ae4a9c168395f4c5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33629 } } }
I20260810 14:47:33.077348 7901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [sys.catalog]: This master's current role is: LEADER
I20260810 14:47:33.077414 7902 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [sys.catalog]: This master's current role is: LEADER
I20260810 14:47:33.078343 7906 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:47:33.094604 7906 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:47:33.100145 7894 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260810 14:47:33.113533 7906 catalog_manager.cc:1347] Generated new cluster ID: 2564102485b04885964dcb7d3989dead
I20260810 14:47:33.113770 7906 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:47:33.139750 7906 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:47:33.141813 7906 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:47:33.150666 7906 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5: Generated new TSK 0
I20260810 14:47:33.151762 7906 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:47:33.947039 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.215.148:56975/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:33629
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:43531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:47:34 dist-test-slave-vkjp krb5kdc[7284](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786373254, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260810 14:47:34.396749 7952 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:34.397135 7952 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:34.397365 7952 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:34.409282 7952 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:34.409439 7952 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:34.409680 7952 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:47:34.425448 7952 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:43531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.24.215.148:56975/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:33629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:34.427214 7952 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:34.428891 7952 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:34.446098 7963 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:34.446172 7965 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:34.450088 7952 server_base.cc:1034] running on GCE node
W20260810 14:47:34.450510 7967 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:34.451347 7952 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:34.458277 7952 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:34.459913 7952 hybrid_clock.cc:648] HybridClock initialized: now 1786373254459784 us; error 129 us; skew 500 ppm
I20260810 14:47:34.460399 7952 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:34 dist-test-slave-vkjp krb5kdc[7284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373254, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:34.465052 7952 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:47:34.470705 7952 webserver.cc:460] Webserver started at http://127.24.215.129:40585/ using document root <none> and password file <none>
I20260810 14:47:34.471750 7952 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:34.471956 7952 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:34.472455 7952 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:47:34 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373254, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:47:34.500389 7952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "fded15ca9fe846eba32985bceeaaf877"
format_stamp: "Formatted at 2026-08-10 14:47:34 on dist-test-slave-vkjp"
server_key: "17a2748a4118111c45a80138c9925d7e"
server_key_iv: "ea92d70eab589f051d56a544ffea007c"
server_key_version: "kuduclusterkey@0"
I20260810 14:47:34.501324 7952 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "fded15ca9fe846eba32985bceeaaf877"
format_stamp: "Formatted at 2026-08-10 14:47:34 on dist-test-slave-vkjp"
server_key: "17a2748a4118111c45a80138c9925d7e"
server_key_iv: "ea92d70eab589f051d56a544ffea007c"
server_key_version: "kuduclusterkey@0"
I20260810 14:47:34.507603 7952 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260810 14:47:34.529838 7974 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:34.531729 7952 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.002s
I20260810 14:47:34.531935 7952 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "fded15ca9fe846eba32985bceeaaf877"
format_stamp: "Formatted at 2026-08-10 14:47:34 on dist-test-slave-vkjp"
server_key: "17a2748a4118111c45a80138c9925d7e"
server_key_iv: "ea92d70eab589f051d56a544ffea007c"
server_key_version: "kuduclusterkey@0"
I20260810 14:47:34.532244 7952 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:34.571940 7952 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:34.578150 7952 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:34.578601 7952 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:34.579912 7952 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:47:34.581918 7952 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:47:34.582015 7952 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:34.582154 7952 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:47:34.582223 7952 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 10 14:47:34 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373254, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:47:34.712682 7952 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:34945
I20260810 14:47:34.712950 8114 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:34945 every 8 connection(s)
I20260810 14:47:34.714605 7952 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:47:34.719240 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 7952
I20260810 14:47:34.719466 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:47:34.720155 25438 external_mini_cluster.cc:1428] Setting key 3d885ea06b323b366f822b12e3b87754
Aug 10 14:47:34 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373254, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:47:34.765053 8117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:34.724443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33629 (local address 127.24.215.129:43213)
0810 14:47:34.737165 (+ 12722us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:34.737221 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:34.739741 (+ 2520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:34.740779 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:34.740793 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:34.741515 (+ 722us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:34.742373 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:34.742403 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:34.744223 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:34.744230 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:34.745936 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:34.745954 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:34.747262 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:34.747271 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:34.747622 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:34.748669 (+ 1047us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:34.748705 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:34.751817 (+ 3112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:34.759293 (+ 7476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:34.759307 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:34.759331 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:34.760092 (+ 761us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:34.760620 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:34.760627 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:34.760631 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:34.760904 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:34.761831 (+ 927us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:34.761843 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:34.762407 (+ 564us) client_negotiation.cc:769] Sending connection context
0810 14:47:34.762798 (+ 391us) client_negotiation.cc:240] Negotiation successful
0810 14:47:34.763449 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12382,"thread_start_us":14104,"threads_started":1}
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
I20260810 14:47:34.766113 8118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:34.725066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:43213 (local address 127.24.215.190:33629)
0810 14:47:34.739989 (+ 14923us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:34.739996 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:34.740029 (+ 33us) server_negotiation.cc:407] Connection header received
0810 14:47:34.740100 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:34.740108 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:34.740201 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:34.740314 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:34.742611 (+ 2297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:34.744012 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:34.746110 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:34.746874 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:34.747006 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:34.752107 (+ 5101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:34.752117 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:34.752124 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:34.752182 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:34.759004 (+ 6822us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:34.760268 (+ 1264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:34.760274 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:34.760278 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:34.760451 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:34.761075 (+ 624us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:34.761083 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:34.761090 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:34.761561 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:34.761734 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:34.765453 (+ 3719us) server_negotiation.cc:299] Negotiation successful
0810 14:47:34.765731 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14776,"thread_start_us":15061,"threads_started":1}
I20260810 14:47:34.767586 8115 heartbeater.cc:347] Connected to a master server at 127.24.215.190:33629
I20260810 14:47:34.767972 8115 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
I20260810 14:47:34.768887 8115 heartbeater.cc:511] Master 127.24.215.190:33629 requested a full tablet report, sending...
I20260810 14:47:34.771864 7852 ts_manager.cc:194] Registered new tserver with Master: fded15ca9fe846eba32985bceeaaf877 (127.24.215.129:34945)
I20260810 14:47:34.774330 7852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:43213
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:47:34.813310 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.215.148:56975/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:33629
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:43531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:47:35 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373254, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
W20260810 14:47:35.276117 8122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:35.276464 8122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:35.276654 8122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:35.284662 8122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:35.284811 8122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:35.285007 8122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:47:35.295912 8122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:43531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.24.215.148:56975/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:33629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:35.297886 8122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:35.299671 8122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:35.328905 8135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:35.329852 8131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:35.331683 8137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:35.331678 8122 server_base.cc:1034] running on GCE node
I20260810 14:47:35.332477 8122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:35.333532 8122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:35.334844 8122 hybrid_clock.cc:648] HybridClock initialized: now 1786373255334791 us; error 41 us; skew 500 ppm
I20260810 14:47:35.335335 8122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:35 dist-test-slave-vkjp krb5kdc[7284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373255, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:35.338737 8122 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:47:35.340380 8122 webserver.cc:460] Webserver started at http://127.24.215.130:37981/ using document root <none> and password file <none>
I20260810 14:47:35.341131 8122 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:35.341287 8122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:35.341614 8122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:47:35 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373255, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:47:35.356428 8122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "575d52fff97f423f8ff271f980d68843"
format_stamp: "Formatted at 2026-08-10 14:47:35 on dist-test-slave-vkjp"
server_key: "ad1d08849101d7088f1a030a7bbe3c70"
server_key_iv: "1343f07a52952f57d261e4e6d5bbd994"
server_key_version: "kuduclusterkey@0"
I20260810 14:47:35.357326 8122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "575d52fff97f423f8ff271f980d68843"
format_stamp: "Formatted at 2026-08-10 14:47:35 on dist-test-slave-vkjp"
server_key: "ad1d08849101d7088f1a030a7bbe3c70"
server_key_iv: "1343f07a52952f57d261e4e6d5bbd994"
server_key_version: "kuduclusterkey@0"
I20260810 14:47:35.362818 8122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 14:47:35.377528 8150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:35.379153 8122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:47:35.379354 8122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "575d52fff97f423f8ff271f980d68843"
format_stamp: "Formatted at 2026-08-10 14:47:35 on dist-test-slave-vkjp"
server_key: "ad1d08849101d7088f1a030a7bbe3c70"
server_key_iv: "1343f07a52952f57d261e4e6d5bbd994"
server_key_version: "kuduclusterkey@0"
I20260810 14:47:35.379601 8122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:35.400570 8122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:35.404345 8122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:35.404667 8122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:35.405681 8122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:47:35.407058 8122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:47:35.407188 8122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:35.407342 8122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:47:35.407423 8122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:35.441123 8122 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:41273
I20260810 14:47:35.441143 8288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:41273 every 8 connection(s)
I20260810 14:47:35.442481 8122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:47:35.448516 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 8122
I20260810 14:47:35.448700 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:47:35.449167 25438 external_mini_cluster.cc:1428] Setting key 873722aebb2bfd22a53029205194165a
Aug 10 14:47:35 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373255, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:47:35.465276 8291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:35.445206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:35287 (local address 127.24.215.190:33629)
0810 14:47:35.445667 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:35.445672 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:35.446657 (+ 985us) server_negotiation.cc:407] Connection header received
0810 14:47:35.448577 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:35.448584 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:35.448687 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:35.448826 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:35.450555 (+ 1729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:35.451467 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:35.453942 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:35.454330 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:35.454429 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:35.458844 (+ 4415us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:35.458849 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:35.458852 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:35.458884 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:35.461741 (+ 2857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:35.462477 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:35.462484 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:35.462489 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:35.462642 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:35.463132 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:35.463139 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:35.463145 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:35.463566 (+ 421us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:35.463733 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:35.464536 (+ 803us) server_negotiation.cc:299] Negotiation successful
0810 14:47:35.464800 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":223,"threads_started":1}
I20260810 14:47:35.465955 8292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:35.445789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33629 (local address 127.24.215.130:35287)
0810 14:47:35.446487 (+ 698us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:35.446528 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:35.448278 (+ 1750us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:35.449076 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:35.449084 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:35.449573 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:35.450365 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:35.450392 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:35.451633 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:35.451638 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:35.453782 (+ 2144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:35.453813 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:35.454559 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:35.454569 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:35.454903 (+ 334us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:35.455674 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:35.455709 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:35.458595 (+ 2886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:35.461894 (+ 3299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:35.461901 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:35.461914 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:35.462345 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:35.462764 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:35.462767 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:35.462769 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:35.462964 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:35.463697 (+ 733us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:35.463721 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:35.464167 (+ 446us) client_negotiation.cc:769] Sending connection context
0810 14:47:35.464401 (+ 234us) client_negotiation.cc:240] Negotiation successful
0810 14:47:35.464789 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":283,"threads_started":1}
I20260810 14:47:35.467561 8289 heartbeater.cc:347] Connected to a master server at 127.24.215.190:33629
I20260810 14:47:35.467898 8289 heartbeater.cc:464] Registering TS with master...
I20260810 14:47:35.468758 8289 heartbeater.cc:511] Master 127.24.215.190:33629 requested a full tablet report, sending...
I20260810 14:47:35.470461 7852 ts_manager.cc:194] Registered new tserver with Master: 575d52fff97f423f8ff271f980d68843 (127.24.215.130:41273)
I20260810 14:47:35.471352 7852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:35287
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:47:35.512503 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.215.148:56975/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:33629
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:43531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:47:35.778824 8115 heartbeater.cc:503] Master 127.24.215.190:33629 was elected leader, sending a full tablet report...
W20260810 14:47:35.882859 8295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:35.883212 8295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:35.883415 8295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:35.891378 8295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:35.891598 8295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:35.891808 8295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:47:35.902642 8295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:43531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.24.215.148:56975/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:33629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:35.904109 8295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:35.905478 8295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:35.921445 8302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:35.921417 8305 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:35.922580 8307 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:35.923667 8295 server_base.cc:1034] running on GCE node
I20260810 14:47:35.924409 8295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:35.925853 8295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:35.927218 8295 hybrid_clock.cc:648] HybridClock initialized: now 1786373255927009 us; error 93 us; skew 500 ppm
I20260810 14:47:35.927620 8295 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:35 dist-test-slave-vkjp krb5kdc[7284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373255, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:35.931981 8295 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:47:35.934098 8295 webserver.cc:460] Webserver started at http://127.24.215.131:33859/ using document root <none> and password file <none>
I20260810 14:47:35.935514 8295 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:35.935695 8295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:35.936138 8295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 10 14:47:35 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373255, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:47:35.954849 8295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "4ce517ea3cc64e5ab64245054ec43572"
format_stamp: "Formatted at 2026-08-10 14:47:35 on dist-test-slave-vkjp"
server_key: "7c44b2c1969f57d0e9f6a7221a64fc7b"
server_key_iv: "0fb88b31d78ae1209844f12d1c4fb4d9"
server_key_version: "kuduclusterkey@0"
I20260810 14:47:35.956090 8295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "4ce517ea3cc64e5ab64245054ec43572"
format_stamp: "Formatted at 2026-08-10 14:47:35 on dist-test-slave-vkjp"
server_key: "7c44b2c1969f57d0e9f6a7221a64fc7b"
server_key_iv: "0fb88b31d78ae1209844f12d1c4fb4d9"
server_key_version: "kuduclusterkey@0"
I20260810 14:47:35.964108 8295 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.003s sys 0.004s
I20260810 14:47:35.983150 8316 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:35.984880 8295 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:47:35.985097 8295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "4ce517ea3cc64e5ab64245054ec43572"
format_stamp: "Formatted at 2026-08-10 14:47:35 on dist-test-slave-vkjp"
server_key: "7c44b2c1969f57d0e9f6a7221a64fc7b"
server_key_iv: "0fb88b31d78ae1209844f12d1c4fb4d9"
server_key_version: "kuduclusterkey@0"
I20260810 14:47:35.985383 8295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:36.008667 8295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:36.013278 8295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:36.013613 8295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:36.015009 8295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:47:36.016707 8295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:47:36.016862 8295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:36.017031 8295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:47:36.017133 8295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:36.056754 8295 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:39453
I20260810 14:47:36.056790 8455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:39453 every 8 connection(s)
I20260810 14:47:36.058076 8295 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:47:36.062739 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 8295
I20260810 14:47:36.062928 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:47:36.063486 25438 external_mini_cluster.cc:1428] Setting key 566e98ebbcb57dfac3dc8d08304ed651
Aug 10 14:47:36 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373255, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:47:36.078220 8458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:36.060866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:51461 (local address 127.24.215.190:33629)
0810 14:47:36.061349 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:36.061354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:36.062049 (+ 695us) server_negotiation.cc:407] Connection header received
0810 14:47:36.064105 (+ 2056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:36.064110 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:36.064190 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:36.064320 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:36.065896 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:36.066817 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:36.068408 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:36.068827 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:36.068962 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:36.072246 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:36.072255 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:36.072265 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:36.072309 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:36.075024 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:36.075823 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:36.075829 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:36.075839 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:36.075962 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:36.076410 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:36.076414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:36.076416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:36.076684 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:36.076839 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:36.077618 (+ 779us) server_negotiation.cc:299] Negotiation successful
0810 14:47:36.077839 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":282,"threads_started":1}
I20260810 14:47:36.079238 8459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:36.061175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33629 (local address 127.24.215.131:51461)
0810 14:47:36.061881 (+ 706us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:36.061922 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:36.063811 (+ 1889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:36.064562 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:36.064570 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:36.065098 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:36.065720 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:36.065742 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:36.066985 (+ 1243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:36.066989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:36.068260 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:36.068273 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:36.069063 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:36.069070 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:36.069298 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:36.069980 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:36.070002 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:36.072039 (+ 2037us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:36.075217 (+ 3178us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:36.075226 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:36.075241 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:36.075697 (+ 456us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:36.076077 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:36.076082 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:36.076086 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:36.076281 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:36.076815 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:36.076823 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:36.077179 (+ 356us) client_negotiation.cc:769] Sending connection context
0810 14:47:36.077522 (+ 343us) client_negotiation.cc:240] Negotiation successful
0810 14:47:36.078025 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":301,"threads_started":1}
I20260810 14:47:36.080564 8456 heartbeater.cc:347] Connected to a master server at 127.24.215.190:33629
I20260810 14:47:36.080892 8456 heartbeater.cc:464] Registering TS with master...
I20260810 14:47:36.081585 8456 heartbeater.cc:511] Master 127.24.215.190:33629 requested a full tablet report, sending...
I20260810 14:47:36.082931 7852 ts_manager.cc:194] Registered new tserver with Master: 4ce517ea3cc64e5ab64245054ec43572 (127.24.215.131:39453)
I20260810 14:47:36.083827 7852 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:51461
I20260810 14:47:36.091027 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:47:36.092934 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 7952
Aug 10 14:47:36 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:47:36.108781 8462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:36.093810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55814 (local address 127.24.215.129:34945)
0810 14:47:36.094435 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:36.094444 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:36.094468 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:47:36.094531 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:36.094538 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:36.094715 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:36.094844 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:36.096395 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:36.097467 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:36.098807 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:36.099389 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:36.099489 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:36.103723 (+ 4234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:36.103733 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:36.103746 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:36.103780 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:36.106364 (+ 2584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:36.106934 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:36.106950 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:36.106952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:36.107143 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:36.107468 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:36.107471 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:36.107473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:36.107789 (+ 316us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:36.108064 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:36.108316 (+ 252us) server_negotiation.cc:299] Negotiation successful
0810 14:47:36.108510 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":476,"thread_start_us":327,"threads_started":1}
I20260810 14:47:36.110049 8109 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:36.192816 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 7952
I20260810 14:47:36.213507 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 8122
Aug 10 14:47:36 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:47:36.234212 8465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:36.214571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40106 (local address 127.24.215.130:41273)
0810 14:47:36.215290 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:36.215297 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:36.215325 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:47:36.215418 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:36.215425 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:36.215638 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:36.215771 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:36.217006 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:36.217989 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:36.219564 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:36.220111 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:36.220211 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:36.228743 (+ 8532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:36.228755 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:36.228770 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:36.228809 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:36.231630 (+ 2821us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:36.232293 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:36.232300 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:36.232302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:36.232435 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:36.232761 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:36.232764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:36.232766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:36.233112 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:36.233342 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:36.233602 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 14:47:36.233801 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":344,"threads_started":1}
I20260810 14:47:36.235755 8284 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:36.318882 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 8122
I20260810 14:47:36.339480 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 8295
Aug 10 14:47:36 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373252, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:47:36.356979 8467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:36.340560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56720 (local address 127.24.215.131:39453)
0810 14:47:36.341192 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:36.341199 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:36.341227 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:47:36.341295 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:36.341302 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:36.341507 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:36.341637 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:36.342901 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:36.343959 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:36.345602 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:36.346155 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:36.346250 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:36.351032 (+ 4782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:36.351042 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:36.351058 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:36.351160 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:36.354127 (+ 2967us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:36.354830 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:36.354836 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:36.354838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:36.354968 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:36.355448 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:36.355452 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:36.355454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:36.355793 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:36.356050 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:36.356365 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 14:47:36.356564 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":491,"thread_start_us":316,"threads_started":1}
I20260810 14:47:36.358290 8450 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:36.441555 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 8295
I20260810 14:47:36.461871 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 7809
I20260810 14:47:36.463241 7888 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:36.570120 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 7809
I20260810 14:47:36.598490 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:33629
--webserver_interface=127.24.215.190
--webserver_port=37401
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:43531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:33629
--ranger_config_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.215.148:56975/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.215.148:56975/kms with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:47:36.936643 8470 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:36.936919 8470 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:36.937040 8470 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:36.944458 8470 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:47:36.944566 8470 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:36.944679 8470 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:36.944785 8470 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:47:36.944875 8470 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:47:36.955119 8470 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:43531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.215.148:56975/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:33629
--ranger_config_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:33629
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=37401
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:36.956403 8470 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:36.957772 8470 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:36.969889 8478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:36.969977 8477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:36.969965 8481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:36.970728 8470 server_base.cc:1034] running on GCE node
I20260810 14:47:36.971567 8470 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:36.972991 8470 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:36.974220 8470 hybrid_clock.cc:648] HybridClock initialized: now 1786373256974155 us; error 50 us; skew 500 ppm
I20260810 14:47:36.974577 8470 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:36 dist-test-slave-vkjp krb5kdc[7284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373256, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:36.977624 8470 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:47:36.979010 8470 webserver.cc:460] Webserver started at http://127.24.215.190:37401/ using document root <none> and password file <none>
I20260810 14:47:36.979678 8470 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:36.979830 8470 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 14:47:37 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373256, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:47:37.036242 8470 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:47:37.039317 8491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:37.040659 8470 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:47:37.040875 8470 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "9f12f4550b7240a8ae4a9c168395f4c5"
format_stamp: "Formatted at 2026-08-10 14:47:32 on dist-test-slave-vkjp"
server_key: "fd513bbbb730037f87add7c8aa723b5c"
server_key_iv: "340a7843ce1fa86d0dfe71e10da6dafa"
server_key_version: "kuduclusterkey@0"
I20260810 14:47:37.041864 8470 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:37.067529 8470 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:37.071223 8470 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:37.071527 8470 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:37.091172 8470 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:33629
I20260810 14:47:37.091195 8550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:33629 every 8 connection(s)
I20260810 14:47:37.092772 8470 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:47:37.095551 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 8470
I20260810 14:47:37.096596 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.215.148:56975/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:34945
--local_ip_for_outbound_sockets=127.24.215.129
--tserver_master_addrs=127.24.215.190:33629
--webserver_port=40585
--webserver_interface=127.24.215.129
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:43531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.215.148:56975/kms with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:47:37.102380 8551 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:47:37.107612 8551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5: Bootstrap starting.
I20260810 14:47:37.126607 8551 log.cc:824] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5: Log is configured to *not* fsync() on all Append() calls
I20260810 14:47:37.135505 8551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:47:37.136068 8551 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5: Bootstrap complete.
I20260810 14:47:37.145375 8551 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f12f4550b7240a8ae4a9c168395f4c5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33629 } }
I20260810 14:47:37.146349 8551 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f12f4550b7240a8ae4a9c168395f4c5, State: Initialized, Role: FOLLOWER
I20260810 14:47:37.147166 8551 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f12f4550b7240a8ae4a9c168395f4c5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33629 } }
I20260810 14:47:37.147399 8551 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:47:37.147547 8551 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:47:37.147737 8551 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:47:37.151564 8551 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f12f4550b7240a8ae4a9c168395f4c5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33629 } }
I20260810 14:47:37.152067 8551 leader_election.cc:302] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9f12f4550b7240a8ae4a9c168395f4c5; no voters:
I20260810 14:47:37.152729 8551 leader_election.cc:288] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 14:47:37.152879 8555 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 14:47:37.154150 8555 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [term 2 LEADER]: Becoming Leader. State: Replica: 9f12f4550b7240a8ae4a9c168395f4c5, State: Running, Role: LEADER
I20260810 14:47:37.154770 8555 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f12f4550b7240a8ae4a9c168395f4c5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33629 } }
I20260810 14:47:37.155110 8551 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:47:37.157490 8556 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "9f12f4550b7240a8ae4a9c168395f4c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f12f4550b7240a8ae4a9c168395f4c5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33629 } } }
I20260810 14:47:37.157910 8556 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [sys.catalog]: This master's current role is: LEADER
I20260810 14:47:37.159710 8560 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:47:37.159603 8557 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9f12f4550b7240a8ae4a9c168395f4c5. Latest consensus state: current_term: 2 leader_uuid: "9f12f4550b7240a8ae4a9c168395f4c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9f12f4550b7240a8ae4a9c168395f4c5" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33629 } } }
I20260810 14:47:37.160035 8557 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5 [sys.catalog]: This master's current role is: LEADER
I20260810 14:47:37.166714 8560 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:47:37.168236 8560 catalog_manager.cc:1259] Loaded cluster ID: 2564102485b04885964dcb7d3989dead
I20260810 14:47:37.168332 8560 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:47:37.172235 8560 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:47:37.173655 8560 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5: Loaded TSK: 0
I20260810 14:47:37.175645 8560 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:47:37.184832 8551 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260810 14:47:37.535342 8553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:37.535858 8553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:37.536062 8553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:37.546870 8553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:37.547109 8553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:37.547359 8553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:47:37.561582 8553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:43531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.215.148:56975/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:34945
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=40585
--tserver_master_addrs=127.24.215.190:33629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:37.563035 8553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:37.564726 8553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:37.602751 8597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:37.603451 8595 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:37.609218 8553 server_base.cc:1034] running on GCE node
W20260810 14:47:37.609861 8599 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:37.610946 8553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:37.616888 8553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:37.620337 8553 hybrid_clock.cc:648] HybridClock initialized: now 1786373257620280 us; error 45 us; skew 500 ppm
I20260810 14:47:37.620798 8553 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:37 dist-test-slave-vkjp krb5kdc[7284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373257, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:37.625198 8553 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:47:37.638038 8553 webserver.cc:460] Webserver started at http://127.24.215.129:40585/ using document root <none> and password file <none>
I20260810 14:47:37.639158 8553 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:37.639338 8553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 14:47:37 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373257, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:47:37.687619 8553 fs_manager.cc:709] Time spent opening directory manager: real 0.011s user 0.007s sys 0.004s
I20260810 14:47:37.698113 8608 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:37.700366 8553 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.005s sys 0.004s
I20260810 14:47:37.700585 8553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "fded15ca9fe846eba32985bceeaaf877"
format_stamp: "Formatted at 2026-08-10 14:47:34 on dist-test-slave-vkjp"
server_key: "17a2748a4118111c45a80138c9925d7e"
server_key_iv: "ea92d70eab589f051d56a544ffea007c"
server_key_version: "kuduclusterkey@0"
I20260810 14:47:37.702119 8553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:37.726662 8553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:37.732010 8553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:37.732422 8553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:37.739436 8553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:47:37.741799 8553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:47:37.741977 8553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 14:47:37.742169 8553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:47:37.742316 8553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:37.853691 8553 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:34945
I20260810 14:47:37.853739 8747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:34945 every 8 connection(s)
I20260810 14:47:37.855875 8553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:47:37.861145 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 8553
I20260810 14:47:37.861937 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.215.148:56975/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:41273
--local_ip_for_outbound_sockets=127.24.215.130
--tserver_master_addrs=127.24.215.190:33629
--webserver_port=37981
--webserver_interface=127.24.215.130
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:43531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.215.148:56975/kms with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:47:37 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373257, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:47:37.899197 8752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:37.864227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33629 (local address 127.24.215.129:58117)
0810 14:47:37.864974 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:37.865010 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:37.866861 (+ 1851us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:37.868530 (+ 1669us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:37.868540 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:37.869135 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:37.869697 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:37.869716 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:37.871476 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:37.871481 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:37.873359 (+ 1878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:37.873378 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:37.875420 (+ 2042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:37.875430 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:37.875659 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:37.883338 (+ 7679us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:37.883397 (+ 59us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:37.886374 (+ 2977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:37.893394 (+ 7020us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:37.893408 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:37.893437 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:37.894163 (+ 726us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:37.894792 (+ 629us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:37.894799 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:37.894802 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:37.895131 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:37.896380 (+ 1249us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:37.896389 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:37.896842 (+ 453us) client_negotiation.cc:769] Sending connection context
0810 14:47:37.897169 (+ 327us) client_negotiation.cc:240] Negotiation successful
0810 14:47:37.897765 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":747,"threads_started":1}
I20260810 14:47:37.899415 8750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:37.859994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:58117 (local address 127.24.215.190:33629)
0810 14:47:37.864388 (+ 4394us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:37.864404 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:37.865154 (+ 750us) server_negotiation.cc:407] Connection header received
0810 14:47:37.867521 (+ 2367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:37.867535 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:37.867979 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:37.868380 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:37.869914 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:37.871303 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:37.873558 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:37.874321 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:37.874528 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:37.886760 (+ 12232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:37.886777 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:37.886805 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:37.886876 (+ 71us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:37.893181 (+ 6305us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:37.894386 (+ 1205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:37.894398 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:37.894407 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:37.894625 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:37.895277 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:37.895285 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:37.895288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:37.896070 (+ 782us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:37.896268 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:37.897500 (+ 1232us) server_negotiation.cc:299] Negotiation successful
0810 14:47:37.897972 (+ 472us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4111,"thread_start_us":4407,"threads_started":1}
I20260810 14:47:37.903528 8748 heartbeater.cc:347] Connected to a master server at 127.24.215.190:33629
I20260810 14:47:37.903868 8748 heartbeater.cc:464] Registering TS with master...
I20260810 14:47:37.904716 8748 heartbeater.cc:511] Master 127.24.215.190:33629 requested a full tablet report, sending...
W20260810 14:47:37.908712 8748 heartbeater.cc:644] Failed to heartbeat to 127.24.215.190:33629 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 14:47:37.908979 8748 heartbeater.cc:464] Registering TS with master...
I20260810 14:47:37.909292 8748 heartbeater.cc:511] Master 127.24.215.190:33629 requested a full tablet report, sending...
I20260810 14:47:37.910693 8748 heartbeater.cc:464] Registering TS with master...
I20260810 14:47:37.910989 8748 heartbeater.cc:511] Master 127.24.215.190:33629 requested a full tablet report, sending...
W20260810 14:47:37.912249 8748 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:47:38.317000 8753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:38.317444 8753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:38.317667 8753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:38.327780 8753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:38.327927 8753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:38.328123 8753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:47:38.339908 8753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:43531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.215.148:56975/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:41273
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=37981
--tserver_master_addrs=127.24.215.190:33629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:38.341735 8753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:38.343520 8753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:38.387719 8769 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:38.390069 8772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:38.407305 8775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:38.408326 8753 server_base.cc:1034] running on GCE node
I20260810 14:47:38.409251 8753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:38.410615 8753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:38.411937 8753 hybrid_clock.cc:648] HybridClock initialized: now 1786373258411871 us; error 45 us; skew 500 ppm
I20260810 14:47:38.412346 8753 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:38 dist-test-slave-vkjp krb5kdc[7284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373258, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:38.416772 8753 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:47:38.418954 8753 webserver.cc:460] Webserver started at http://127.24.215.130:37981/ using document root <none> and password file <none>
I20260810 14:47:38.420020 8753 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:38.420205 8753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 14:47:38 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373258, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:47:38.445616 8753 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:47:38.451443 8783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:38.453411 8753 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260810 14:47:38.453599 8753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "575d52fff97f423f8ff271f980d68843"
format_stamp: "Formatted at 2026-08-10 14:47:35 on dist-test-slave-vkjp"
server_key: "ad1d08849101d7088f1a030a7bbe3c70"
server_key_iv: "1343f07a52952f57d261e4e6d5bbd994"
server_key_version: "kuduclusterkey@0"
I20260810 14:47:38.455109 8753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
Aug 10 14:47:38 dist-test-slave-vkjp krb5kdc[7284](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786373258, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:38.509604 8753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:38.514955 8753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:38.515393 8753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:38.516737 8753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:47:38.519037 8753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:47:38.519222 8753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 14:47:38.519382 8753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:47:38.519493 8753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:38.581076 8753 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:41273
I20260810 14:47:38.581174 8917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:41273 every 8 connection(s)
I20260810 14:47:38.582993 8753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:47:38.583894 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 8753
I20260810 14:47:38.584903 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.24.215.148:56975/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:39453
--local_ip_for_outbound_sockets=127.24.215.131
--tserver_master_addrs=127.24.215.190:33629
--webserver_port=33859
--webserver_interface=127.24.215.131
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:43531
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.24.215.148:56975/kms with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:47:38 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373258, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:47:38.657670 8922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:38.615336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:58941 (local address 127.24.215.190:33629)
0810 14:47:38.626736 (+ 11400us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:38.626745 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:38.635207 (+ 8462us) server_negotiation.cc:407] Connection header received
0810 14:47:38.636084 (+ 877us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:38.636090 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:38.636188 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:38.636300 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:38.638706 (+ 2406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:38.639948 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:38.643010 (+ 3062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:38.643709 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:38.643817 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:38.648656 (+ 4839us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:38.648664 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:38.648669 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:38.648719 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:38.652774 (+ 4055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:38.653881 (+ 1107us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:38.653888 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:38.653892 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:38.654052 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:38.655013 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:38.655018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:38.655023 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:38.655633 (+ 610us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:38.655758 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:38.657023 (+ 1265us) server_negotiation.cc:299] Negotiation successful
0810 14:47:38.657271 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11238,"thread_start_us":10919,"threads_started":1}
I20260810 14:47:38.660619 8921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:38.614642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33629 (local address 127.24.215.130:58941)
0810 14:47:38.632634 (+ 17992us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:38.632736 (+ 102us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:38.635517 (+ 2781us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:38.636823 (+ 1306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:38.636838 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:38.637541 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:38.638457 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:38.638487 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:38.640181 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:38.640186 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:38.642811 (+ 2625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:38.642828 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:38.644202 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:38.644211 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:38.644518 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:38.645370 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:38.645416 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:38.648375 (+ 2959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:38.653020 (+ 4645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:38.653032 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:38.653051 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:38.653697 (+ 646us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:38.654229 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:38.654234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:38.654237 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:38.654869 (+ 632us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:38.655916 (+ 1047us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:38.655924 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:38.656396 (+ 472us) client_negotiation.cc:769] Sending connection context
0810 14:47:38.658359 (+ 1963us) client_negotiation.cc:240] Negotiation successful
0810 14:47:38.658986 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17614,"thread_start_us":17360,"threads_started":1}
I20260810 14:47:38.662886 8918 heartbeater.cc:347] Connected to a master server at 127.24.215.190:33629
I20260810 14:47:38.663290 8918 heartbeater.cc:464] Registering TS with master...
I20260810 14:47:38.664261 8918 heartbeater.cc:511] Master 127.24.215.190:33629 requested a full tablet report, sending...
I20260810 14:47:38.666509 8512 ts_manager.cc:194] Registered new tserver with Master: 575d52fff97f423f8ff271f980d68843 (127.24.215.130:41273)
I20260810 14:47:38.668903 8512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:58941
Aug 10 14:47:38 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373258, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:47:38.912451 8748 heartbeater.cc:464] Registering TS with master...
I20260810 14:47:38.912966 8748 heartbeater.cc:511] Master 127.24.215.190:33629 requested a full tablet report, sending...
I20260810 14:47:38.914268 8512 ts_manager.cc:194] Registered new tserver with Master: fded15ca9fe846eba32985bceeaaf877 (127.24.215.129:34945)
I20260810 14:47:38.915494 8512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:58117
W20260810 14:47:39.081641 8919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:39.082172 8919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:39.082417 8919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:39.094050 8919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:39.094266 8919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:39.094542 8919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:47:39.110481 8919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:43531
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.24.215.148:56975/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:39453
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=33859
--tserver_master_addrs=127.24.215.190:33629
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:39.112519 8919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:39.114326 8919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:39.148953 8929 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:39.157025 8928 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:39.157039 8933 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:39.157496 8919 server_base.cc:1034] running on GCE node
I20260810 14:47:39.158651 8919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:39.160051 8919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:39.161362 8919 hybrid_clock.cc:648] HybridClock initialized: now 1786373259161237 us; error 108 us; skew 500 ppm
I20260810 14:47:39.161834 8919 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:39 dist-test-slave-vkjp krb5kdc[7284](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373259, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:39.171603 8919 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:47:39.195392 8919 webserver.cc:460] Webserver started at http://127.24.215.131:33859/ using document root <none> and password file <none>
I20260810 14:47:39.196434 8919 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:39.196592 8919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 10 14:47:39 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373259, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
Aug 10 14:47:39 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373258, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for HTTP/127.24.215.148@KRBTEST.COM
I20260810 14:47:39.229983 8919 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 14:47:39.233700 8942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:39.235816 8919 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:47:39.236102 8919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "4ce517ea3cc64e5ab64245054ec43572"
format_stamp: "Formatted at 2026-08-10 14:47:35 on dist-test-slave-vkjp"
server_key: "7c44b2c1969f57d0e9f6a7221a64fc7b"
server_key_iv: "0fb88b31d78ae1209844f12d1c4fb4d9"
server_key_version: "kuduclusterkey@0"
I20260810 14:47:39.237977 8919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:39.259889 8919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:39.265777 8919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:39.266247 8919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:39.267710 8919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:47:39.269585 8919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:47:39.269759 8919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:39.269958 8919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:47:39.270133 8919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:39.430001 8919 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:39453
I20260810 14:47:39.430069 9068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:39453 every 8 connection(s)
I20260810 14:47:39.431697 8919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:47:39.435454 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 8919
Aug 10 14:47:39 dist-test-slave-vkjp krb5kdc[7284](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373259, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:47:39.478557 9074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:39.437019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59212 (local address 127.24.215.190:33629)
0810 14:47:39.454663 (+ 17644us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:39.454669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:39.454692 (+ 23us) server_negotiation.cc:407] Connection header received
0810 14:47:39.454774 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:39.454780 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:39.454887 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:39.455059 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:39.458129 (+ 3070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.459330 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.462059 (+ 2729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.462750 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.462892 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:39.467752 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:39.467759 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:39.467765 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:39.467826 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:39.474346 (+ 6520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:39.476071 (+ 1725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:39.476077 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:39.476080 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:39.476285 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:39.476741 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:39.476746 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:39.476750 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:39.477120 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:39.477819 (+ 699us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:39.477956 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 14:47:39.478175 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17518,"thread_start_us":14172,"threads_started":1}
I20260810 14:47:39.484753 9072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:39.436588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:54983 (local address 127.24.215.190:33629)
0810 14:47:39.450943 (+ 14355us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:39.450950 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:39.457148 (+ 6198us) server_negotiation.cc:407] Connection header received
0810 14:47:39.458731 (+ 1583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:39.458738 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:39.458815 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:39.458897 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:39.465345 (+ 6448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.466646 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.468632 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.469245 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.469394 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:39.473881 (+ 4487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:39.473889 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:39.473894 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:39.473940 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:39.477532 (+ 3592us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:39.480919 (+ 3387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:39.480925 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:39.480928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:39.481114 (+ 186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:39.482694 (+ 1580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:39.482700 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:39.482703 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:39.483043 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:39.483221 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:39.484148 (+ 927us) server_negotiation.cc:299] Negotiation successful
0810 14:47:39.484377 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14221,"thread_start_us":14097,"threads_started":1}
I20260810 14:47:39.485975 9075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:39.437151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33629 (local address 127.24.215.131:54983)
0810 14:47:39.453596 (+ 16445us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:39.453648 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:39.458367 (+ 4719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:39.461628 (+ 3261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:39.461643 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:39.462309 (+ 666us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:39.465117 (+ 2808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:39.465150 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.466844 (+ 1694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.466851 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:39.468470 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:39.468483 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.469512 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.469524 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:39.470067 (+ 543us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:39.470789 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:39.470827 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:39.473600 (+ 2773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:39.479424 (+ 5824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:39.479432 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:39.479450 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:39.480042 (+ 592us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:39.482206 (+ 2164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:39.482211 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:39.482215 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:39.482530 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:39.483217 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:39.483226 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:39.483666 (+ 440us) client_negotiation.cc:769] Sending connection context
0810 14:47:39.483995 (+ 329us) client_negotiation.cc:240] Negotiation successful
0810 14:47:39.484550 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16093,"thread_start_us":16310,"threads_started":1}
I20260810 14:47:39.487731 9070 heartbeater.cc:347] Connected to a master server at 127.24.215.190:33629
I20260810 14:47:39.488085 9070 heartbeater.cc:464] Registering TS with master...
I20260810 14:47:39.489006 9070 heartbeater.cc:511] Master 127.24.215.190:33629 requested a full tablet report, sending...
I20260810 14:47:39.490830 8512 ts_manager.cc:194] Registered new tserver with Master: 4ce517ea3cc64e5ab64245054ec43572 (127.24.215.131:39453)
I20260810 14:47:39.492046 8512 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:54983
I20260810 14:47:39.494293 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:47:39.520504 9072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:39.498720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59226 (local address 127.24.215.190:33629)
0810 14:47:39.498902 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:39.498908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:39.501837 (+ 2929us) server_negotiation.cc:407] Connection header received
0810 14:47:39.502025 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:39.502032 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:39.502117 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:39.502229 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:39.504355 (+ 2126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.505456 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.508640 (+ 3184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.509276 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.509397 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:39.513754 (+ 4357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:39.513761 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:39.513766 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:39.513809 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:39.517463 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:39.518400 (+ 937us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:39.518406 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:39.518410 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:39.518554 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:39.519012 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:39.519017 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:39.519021 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:39.519444 (+ 423us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:39.519689 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:39.520048 (+ 359us) server_negotiation.cc:299] Negotiation successful
0810 14:47:39.520229 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:47:39.526589 8512 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:47:39.537884 8512 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:47:39.576759 9090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:39.557162 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:41273 (local address 127.0.0.1:40120)
0810 14:47:39.566146 (+ 8984us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:39.566193 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:39.566357 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:39.567899 (+ 1542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:39.567907 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:39.567952 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:47:39.568431 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:39.568449 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.571175 (+ 2726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.571181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:39.573760 (+ 2579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:39.573779 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.575788 (+ 2009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.575794 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:39.576018 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:39.576075 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 14:47:39.576195 (+ 120us) client_negotiation.cc:240] Negotiation successful
0810 14:47:39.576392 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8719,"thread_start_us":8574,"threads_started":1}
I20260810 14:47:39.578080 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:39.557394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40120 (local address 127.24.215.130:41273)
0810 14:47:39.567195 (+ 9801us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:39.567217 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:39.567252 (+ 35us) server_negotiation.cc:407] Connection header received
0810 14:47:39.567362 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:39.567370 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:39.567644 (+ 274us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:39.568692 (+ 1048us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:47:39.568920 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.570983 (+ 2063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.574013 (+ 3030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.575546 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.577303 (+ 1757us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:39.577493 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:39.577565 (+ 72us) server_negotiation.cc:299] Negotiation successful
0810 14:47:39.577740 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9390,"thread_start_us":9273,"threads_started":1}
I20260810 14:47:39.587561 8840 tablet_service.cc:1467] Processing CreateTablet for tablet 5006c80a989d4d27ba8e8f3e313dd87a (DEFAULT_TABLE table=test-table [id=21561ca9fbbb470bbc8a4447133e9221]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:47:39.589398 8840 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5006c80a989d4d27ba8e8f3e313dd87a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:47:39.599102 9090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:39.581182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:39453 (local address 127.0.0.1:56728)
0810 14:47:39.581611 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:39.581634 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:39.581787 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:39.587449 (+ 5662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:39.587454 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:39.587482 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:47:39.587722 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:39.587734 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.590401 (+ 2667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.590406 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:39.595696 (+ 5290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:39.595717 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.598231 (+ 2514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.598237 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:39.598471 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:39.598521 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 14:47:39.598603 (+ 82us) client_negotiation.cc:240] Negotiation successful
0810 14:47:39.598771 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274}
I20260810 14:47:39.601126 9094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:39.575198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56728 (local address 127.24.215.131:39453)
0810 14:47:39.586635 (+ 11437us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:39.586643 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:39.586677 (+ 34us) server_negotiation.cc:407] Connection header received
0810 14:47:39.586773 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:39.586781 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:39.587021 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:39.588320 (+ 1299us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:47:39.588663 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.590240 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.596462 (+ 6222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.598025 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.600306 (+ 2281us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:39.600493 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:39.600588 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 14:47:39.600749 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11238,"thread_start_us":11958,"threads_started":1}
I20260810 14:47:39.605417 8995 tablet_service.cc:1467] Processing CreateTablet for tablet 5006c80a989d4d27ba8e8f3e313dd87a (DEFAULT_TABLE table=test-table [id=21561ca9fbbb470bbc8a4447133e9221]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:47:39.607074 8995 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5006c80a989d4d27ba8e8f3e313dd87a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:47:39.607631 9092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:39.567304 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:34945 (local address 127.0.0.1:55826)
0810 14:47:39.574170 (+ 6866us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:39.574198 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:39.585714 (+ 11516us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:39.598600 (+ 12886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:39.598606 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:39.598641 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:47:39.599105 (+ 464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:39.599122 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.602440 (+ 3318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.602446 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:39.604623 (+ 2177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:39.604640 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.606778 (+ 2138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.606787 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:39.606988 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:39.607028 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 14:47:39.607127 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 14:47:39.607280 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6687,"thread_start_us":6997,"threads_started":1}
I20260810 14:47:39.608765 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:39.567655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55826 (local address 127.24.215.129:34945)
0810 14:47:39.597687 (+ 30032us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:39.597696 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:39.597730 (+ 34us) server_negotiation.cc:407] Connection header received
0810 14:47:39.597834 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:39.597843 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:39.598123 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:39.599344 (+ 1221us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:47:39.600567 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.602263 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.604838 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.606479 (+ 1641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.607968 (+ 1489us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:39.608142 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:39.608272 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 14:47:39.608446 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29818,"thread_start_us":30627,"threads_started":1}
I20260810 14:47:39.612846 8666 tablet_service.cc:1467] Processing CreateTablet for tablet 5006c80a989d4d27ba8e8f3e313dd87a (DEFAULT_TABLE table=test-table [id=21561ca9fbbb470bbc8a4447133e9221]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:47:39.614415 8666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5006c80a989d4d27ba8e8f3e313dd87a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:47:39.624316 9100 tablet_bootstrap.cc:492] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843: Bootstrap starting.
I20260810 14:47:39.626116 9102 tablet_bootstrap.cc:492] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572: Bootstrap starting.
I20260810 14:47:39.630143 9102 tablet_bootstrap.cc:654] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572: Neither blocks nor log segments found. Creating new log.
I20260810 14:47:39.630151 9105 tablet_bootstrap.cc:492] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877: Bootstrap starting.
I20260810 14:47:39.631145 9100 tablet_bootstrap.cc:654] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843: Neither blocks nor log segments found. Creating new log.
I20260810 14:47:39.631521 9102 log.cc:824] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572: Log is configured to *not* fsync() on all Append() calls
I20260810 14:47:39.632491 9100 log.cc:824] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843: Log is configured to *not* fsync() on all Append() calls
I20260810 14:47:39.633199 9105 tablet_bootstrap.cc:654] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877: Neither blocks nor log segments found. Creating new log.
I20260810 14:47:39.634375 9105 log.cc:824] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877: Log is configured to *not* fsync() on all Append() calls
I20260810 14:47:39.635540 9102 tablet_bootstrap.cc:492] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572: No bootstrap required, opened a new log
I20260810 14:47:39.635830 9102 ts_tablet_manager.cc:1397] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 14:47:39.635941 9100 tablet_bootstrap.cc:492] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843: No bootstrap required, opened a new log
I20260810 14:47:39.636189 9100 ts_tablet_manager.cc:1397] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.003s
I20260810 14:47:39.637501 9105 tablet_bootstrap.cc:492] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877: No bootstrap required, opened a new log
I20260810 14:47:39.637754 9105 ts_tablet_manager.cc:1397] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 14:47:39.643656 9100 raft_consensus.cc:348] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575d52fff97f423f8ff271f980d68843" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41273 } } peers { permanent_uuid: "fded15ca9fe846eba32985bceeaaf877" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 34945 } } peers { permanent_uuid: "4ce517ea3cc64e5ab64245054ec43572" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39453 } }
I20260810 14:47:39.644038 9100 raft_consensus.cc:374] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:47:39.644148 9100 raft_consensus.cc:729] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 575d52fff97f423f8ff271f980d68843, State: Initialized, Role: FOLLOWER
I20260810 14:47:39.644817 9100 consensus_queue.cc:260] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575d52fff97f423f8ff271f980d68843" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41273 } } peers { permanent_uuid: "fded15ca9fe846eba32985bceeaaf877" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 34945 } } peers { permanent_uuid: "4ce517ea3cc64e5ab64245054ec43572" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39453 } }
I20260810 14:47:39.646032 8918 heartbeater.cc:503] Master 127.24.215.190:33629 was elected leader, sending a full tablet report...
I20260810 14:47:39.646823 9100 ts_tablet_manager.cc:1428] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260810 14:47:39.647977 9102 raft_consensus.cc:348] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575d52fff97f423f8ff271f980d68843" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41273 } } peers { permanent_uuid: "fded15ca9fe846eba32985bceeaaf877" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 34945 } } peers { permanent_uuid: "4ce517ea3cc64e5ab64245054ec43572" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39453 } }
I20260810 14:47:39.648100 9105 raft_consensus.cc:348] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575d52fff97f423f8ff271f980d68843" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41273 } } peers { permanent_uuid: "fded15ca9fe846eba32985bceeaaf877" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 34945 } } peers { permanent_uuid: "4ce517ea3cc64e5ab64245054ec43572" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39453 } }
I20260810 14:47:39.648418 9102 raft_consensus.cc:374] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:47:39.648448 9105 raft_consensus.cc:374] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:47:39.648535 9102 raft_consensus.cc:729] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ce517ea3cc64e5ab64245054ec43572, State: Initialized, Role: FOLLOWER
I20260810 14:47:39.648572 9105 raft_consensus.cc:729] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fded15ca9fe846eba32985bceeaaf877, State: Initialized, Role: FOLLOWER
I20260810 14:47:39.649201 9105 consensus_queue.cc:260] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575d52fff97f423f8ff271f980d68843" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41273 } } peers { permanent_uuid: "fded15ca9fe846eba32985bceeaaf877" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 34945 } } peers { permanent_uuid: "4ce517ea3cc64e5ab64245054ec43572" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39453 } }
I20260810 14:47:39.649148 9102 consensus_queue.cc:260] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575d52fff97f423f8ff271f980d68843" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41273 } } peers { permanent_uuid: "fded15ca9fe846eba32985bceeaaf877" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 34945 } } peers { permanent_uuid: "4ce517ea3cc64e5ab64245054ec43572" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39453 } }
I20260810 14:47:39.650949 9105 ts_tablet_manager.cc:1428] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877: Time spent starting tablet: real 0.013s user 0.010s sys 0.004s
I20260810 14:47:39.651691 9070 heartbeater.cc:503] Master 127.24.215.190:33629 was elected leader, sending a full tablet report...
I20260810 14:47:39.651762 8748 heartbeater.cc:503] Master 127.24.215.190:33629 was elected leader, sending a full tablet report...
I20260810 14:47:39.652267 9102 ts_tablet_manager.cc:1428] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572: Time spent starting tablet: real 0.016s user 0.010s sys 0.004s
W20260810 14:47:39.701260 9071 tablet.cc:2369] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:47:39.865813 8749 tablet.cc:2369] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:47:39.870402 8920 tablet.cc:2369] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:47:39.895202 9109 raft_consensus.cc:482] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:47:39.895514 9109 raft_consensus.cc:504] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575d52fff97f423f8ff271f980d68843" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41273 } } peers { permanent_uuid: "fded15ca9fe846eba32985bceeaaf877" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 34945 } } peers { permanent_uuid: "4ce517ea3cc64e5ab64245054ec43572" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39453 } }
I20260810 14:47:39.897396 9109 leader_election.cc:288] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fded15ca9fe846eba32985bceeaaf877 (127.24.215.129:34945), 4ce517ea3cc64e5ab64245054ec43572 (127.24.215.131:39453)
I20260810 14:47:39.905803 9113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:39.897747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:34945 (local address 127.24.215.130:47639)
0810 14:47:39.898505 (+ 758us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:39.898523 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:39.898682 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:39.899465 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:39.899471 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:39.899506 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:47:39.899911 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:39.899927 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.901154 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.901162 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:39.902981 (+ 1819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:39.902996 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.904931 (+ 1935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.904937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:39.905125 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:39.905171 (+ 46us) client_negotiation.cc:769] Sending connection context
0810 14:47:39.905309 (+ 138us) client_negotiation.cc:240] Negotiation successful
0810 14:47:39.905438 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":555,"thread_start_us":382,"threads_started":1}
I20260810 14:47:39.905819 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:39.898280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:47639 (local address 127.24.215.129:34945)
0810 14:47:39.899058 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:39.899066 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:39.899128 (+ 62us) server_negotiation.cc:407] Connection header received
0810 14:47:39.899218 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:39.899223 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:39.899319 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:39.899427 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:47:39.900056 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.901031 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.903182 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.904688 (+ 1506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.904886 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:39.905003 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:39.905345 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 14:47:39.905513 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":564}
I20260810 14:47:39.905948 9094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:39.898896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:42947 (local address 127.24.215.131:39453)
0810 14:47:39.899063 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:39.899067 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:39.899627 (+ 560us) server_negotiation.cc:407] Connection header received
0810 14:47:39.899913 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:39.899922 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:39.900024 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:39.900132 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:47:39.900689 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.901916 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.903989 (+ 2073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.904801 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.904964 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:39.905041 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:39.905554 (+ 513us) server_negotiation.cc:299] Negotiation successful
0810 14:47:39.905692 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260810 14:47:39.906611 9114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:39.898680 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:39453 (local address 127.24.215.130:42947)
0810 14:47:39.899515 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:39.899533 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:39.899731 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:39.900139 (+ 408us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:39.900147 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:39.900175 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:47:39.900526 (+ 351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:39.900540 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.902078 (+ 1538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.902083 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:39.903844 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:39.903856 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.905183 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.905189 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:39.905379 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:39.905408 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 14:47:39.906114 (+ 706us) client_negotiation.cc:240] Negotiation successful
0810 14:47:39.906236 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":717,"thread_start_us":876,"threads_started":1}
I20260810 14:47:39.907425 8689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5006c80a989d4d27ba8e8f3e313dd87a" candidate_uuid: "575d52fff97f423f8ff271f980d68843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fded15ca9fe846eba32985bceeaaf877" is_pre_election: true
I20260810 14:47:39.907779 9019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5006c80a989d4d27ba8e8f3e313dd87a" candidate_uuid: "575d52fff97f423f8ff271f980d68843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ce517ea3cc64e5ab64245054ec43572" is_pre_election: true
I20260810 14:47:39.907984 8689 raft_consensus.cc:2454] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575d52fff97f423f8ff271f980d68843 in term 0.
I20260810 14:47:39.908298 9019 raft_consensus.cc:2454] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 575d52fff97f423f8ff271f980d68843 in term 0.
I20260810 14:47:39.908746 8785 leader_election.cc:302] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 575d52fff97f423f8ff271f980d68843, fded15ca9fe846eba32985bceeaaf877; no voters:
I20260810 14:47:39.909145 9109 raft_consensus.cc:2781] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:47:39.909307 9109 raft_consensus.cc:482] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:47:39.909440 9109 raft_consensus.cc:3037] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:47:39.911862 9109 raft_consensus.cc:504] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575d52fff97f423f8ff271f980d68843" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41273 } } peers { permanent_uuid: "fded15ca9fe846eba32985bceeaaf877" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 34945 } } peers { permanent_uuid: "4ce517ea3cc64e5ab64245054ec43572" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39453 } }
I20260810 14:47:39.912459 9109 leader_election.cc:288] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [CANDIDATE]: Term 1 election: Requested vote from peers fded15ca9fe846eba32985bceeaaf877 (127.24.215.129:34945), 4ce517ea3cc64e5ab64245054ec43572 (127.24.215.131:39453)
I20260810 14:47:39.912962 8689 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5006c80a989d4d27ba8e8f3e313dd87a" candidate_uuid: "575d52fff97f423f8ff271f980d68843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fded15ca9fe846eba32985bceeaaf877"
I20260810 14:47:39.913030 9019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5006c80a989d4d27ba8e8f3e313dd87a" candidate_uuid: "575d52fff97f423f8ff271f980d68843" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4ce517ea3cc64e5ab64245054ec43572"
I20260810 14:47:39.913172 8689 raft_consensus.cc:3037] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:47:39.913247 9019 raft_consensus.cc:3037] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:47:39.915572 9019 raft_consensus.cc:2454] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575d52fff97f423f8ff271f980d68843 in term 1.
I20260810 14:47:39.915863 8689 raft_consensus.cc:2454] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 575d52fff97f423f8ff271f980d68843 in term 1.
I20260810 14:47:39.916005 8785 leader_election.cc:302] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4ce517ea3cc64e5ab64245054ec43572, 575d52fff97f423f8ff271f980d68843; no voters:
I20260810 14:47:39.916321 9109 raft_consensus.cc:2781] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:47:39.916778 9109 raft_consensus.cc:686] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [term 1 LEADER]: Becoming Leader. State: Replica: 575d52fff97f423f8ff271f980d68843, State: Running, Role: LEADER
I20260810 14:47:39.917248 9109 consensus_queue.cc:237] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575d52fff97f423f8ff271f980d68843" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41273 } } peers { permanent_uuid: "fded15ca9fe846eba32985bceeaaf877" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 34945 } } peers { permanent_uuid: "4ce517ea3cc64e5ab64245054ec43572" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39453 } }
I20260810 14:47:39.922569 8512 catalog_manager.cc:5495] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 reported cstate change: term changed from 0 to 1, leader changed from <none> to 575d52fff97f423f8ff271f980d68843 (127.24.215.130). New cstate: current_term: 1 leader_uuid: "575d52fff97f423f8ff271f980d68843" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "575d52fff97f423f8ff271f980d68843" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 41273 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fded15ca9fe846eba32985bceeaaf877" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 34945 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4ce517ea3cc64e5ab64245054ec43572" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39453 } health_report { overall_health: UNKNOWN } } }
I20260810 14:47:39.974566 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:39.969120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40128 (local address 127.24.215.130:41273)
0810 14:47:39.969345 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:39.969351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:39.969373 (+ 22us) server_negotiation.cc:407] Connection header received
0810 14:47:39.969598 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:39.969602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:39.969681 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:39.969841 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:47:39.970228 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.970985 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.972353 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:39.972760 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:39.972897 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:39.973434 (+ 537us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:47:39.973958 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:47:39.974134 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:39.974195 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 14:47:39.974273 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:47:39.983582 8689 raft_consensus.cc:1261] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877 [term 1 FOLLOWER]: Refusing update from remote peer 575d52fff97f423f8ff271f980d68843: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:47:39.983630 9019 raft_consensus.cc:1261] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572 [term 1 FOLLOWER]: Refusing update from remote peer 575d52fff97f423f8ff271f980d68843: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:47:39.984639 9115 consensus_queue.cc:1035] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4ce517ea3cc64e5ab64245054ec43572" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39453 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:47:39.985033 9109 consensus_queue.cc:1035] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fded15ca9fe846eba32985bceeaaf877" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 34945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:47:40.013507 8510 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:47:40.013779 8510 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:47:40.017624 8510 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9f12f4550b7240a8ae4a9c168395f4c5: Sending DeleteTablet for 3 replicas of tablet 5006c80a989d4d27ba8e8f3e313dd87a
I20260810 14:47:40.018803 8840 tablet_service.cc:1514] Processing DeleteTablet for tablet 5006c80a989d4d27ba8e8f3e313dd87a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:47:40 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:40120
I20260810 14:47:40.019032 8995 tablet_service.cc:1514] Processing DeleteTablet for tablet 5006c80a989d4d27ba8e8f3e313dd87a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:47:40 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:56728
I20260810 14:47:40.019645 8666 tablet_service.cc:1514] Processing DeleteTablet for tablet 5006c80a989d4d27ba8e8f3e313dd87a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:47:40 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:55826
I20260810 14:47:40.019951 9130 tablet_replica.cc:330] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843: stopping tablet replica
I20260810 14:47:40.020392 9130 raft_consensus.cc:2229] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:47:40.021176 9130 raft_consensus.cc:2258] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:47:40.023396 9130 ts_tablet_manager.cc:1905] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:47:40.026055 9133 tablet_replica.cc:330] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877: stopping tablet replica
I20260810 14:47:40.026525 9133 raft_consensus.cc:2229] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:47:40.027051 9133 raft_consensus.cc:2258] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:47:40.027117 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 8553
I20260810 14:47:40.031528 9130 ts_tablet_manager.cc:1918] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:47:40.031745 9130 log.cc:1196] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/5006c80a989d4d27ba8e8f3e313dd87a
I20260810 14:47:40.032366 9130 ts_tablet_manager.cc:1939] T 5006c80a989d4d27ba8e8f3e313dd87a P 575d52fff97f423f8ff271f980d68843: Deleting consensus metadata
I20260810 14:47:40.034403 8495 catalog_manager.cc:4841] TS 575d52fff97f423f8ff271f980d68843 (127.24.215.130:41273): tablet 5006c80a989d4d27ba8e8f3e313dd87a (table test-table [id=21561ca9fbbb470bbc8a4447133e9221]) successfully deleted
I20260810 14:47:40.035645 9132 tablet_replica.cc:330] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572: stopping tablet replica
I20260810 14:47:40.036108 9132 raft_consensus.cc:2229] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:47:40.036630 9132 raft_consensus.cc:2258] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:47:40.037339 9133 ts_tablet_manager.cc:1905] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:47:40.038738 9132 ts_tablet_manager.cc:1905] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:47:40.044902 9133 ts_tablet_manager.cc:1918] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:47:40.045143 9133 log.cc:1196] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/5006c80a989d4d27ba8e8f3e313dd87a
I20260810 14:47:40.045843 9133 ts_tablet_manager.cc:1939] T 5006c80a989d4d27ba8e8f3e313dd87a P fded15ca9fe846eba32985bceeaaf877: Deleting consensus metadata
I20260810 14:47:40.045994 9132 ts_tablet_manager.cc:1918] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:47:40.046209 9132 log.cc:1196] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/5006c80a989d4d27ba8e8f3e313dd87a
I20260810 14:47:40.046836 9132 ts_tablet_manager.cc:1939] T 5006c80a989d4d27ba8e8f3e313dd87a P 4ce517ea3cc64e5ab64245054ec43572: Deleting consensus metadata
I20260810 14:47:40.047969 8495 catalog_manager.cc:4841] TS fded15ca9fe846eba32985bceeaaf877 (127.24.215.129:34945): tablet 5006c80a989d4d27ba8e8f3e313dd87a (table test-table [id=21561ca9fbbb470bbc8a4447133e9221]) successfully deleted
I20260810 14:47:40.048679 8495 catalog_manager.cc:4841] TS 4ce517ea3cc64e5ab64245054ec43572 (127.24.215.131:39453): tablet 5006c80a989d4d27ba8e8f3e313dd87a (table test-table [id=21561ca9fbbb470bbc8a4447133e9221]) successfully deleted
I20260810 14:47:40.058144 9093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:40.030926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55830 (local address 127.24.215.129:34945)
0810 14:47:40.031146 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:40.031152 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:40.041582 (+ 10430us) server_negotiation.cc:407] Connection header received
0810 14:47:40.042057 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:40.042063 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:40.042166 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:40.042251 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:40.044334 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:40.045546 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:40.047607 (+ 2061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:40.048268 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:40.048371 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:40.051290 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:40.051307 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:40.051327 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:40.051374 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:40.055157 (+ 3783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:40.055825 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:40.055837 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:40.055843 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:40.056019 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:40.056453 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:40.056458 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:40.056461 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:40.056912 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:40.057249 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:40.057486 (+ 237us) server_negotiation.cc:299] Negotiation successful
0810 14:47:40.057723 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 14:47:40.058944 8740 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:40.166225 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 8553
I20260810 14:47:40.191807 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 8753
I20260810 14:47:40.206725 9091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:40.192749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40142 (local address 127.24.215.130:41273)
0810 14:47:40.192941 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:40.192948 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:40.192968 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:47:40.193184 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:40.193190 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:40.193265 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:40.193390 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:40.194729 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:40.195684 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:40.196975 (+ 1291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:40.197614 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:40.197758 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:40.200479 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:40.200495 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:40.200523 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:40.200565 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:40.203804 (+ 3239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:40.204508 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:40.204520 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:40.204528 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:40.204687 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:40.205081 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:40.205087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:40.205091 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:40.205461 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:40.205809 (+ 348us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:40.206112 (+ 303us) server_negotiation.cc:299] Negotiation successful
0810 14:47:40.206335 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 14:47:40.207671 8913 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:40.324249 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 8753
I20260810 14:47:40.348089 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 8919
I20260810 14:47:40.362154 9094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:40.348903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56730 (local address 127.24.215.131:39453)
0810 14:47:40.349086 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:40.349093 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:40.349192 (+ 99us) server_negotiation.cc:407] Connection header received
0810 14:47:40.349412 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:40.349417 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:40.349511 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:40.349620 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:40.350986 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:40.351784 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:40.353116 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:40.353532 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:40.353676 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:40.356427 (+ 2751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:40.356453 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:40.356469 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:40.356501 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:40.359242 (+ 2741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:40.359918 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:40.359927 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:40.359931 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:40.360088 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:40.360500 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:40.360505 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:40.360508 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:40.360863 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:40.361189 (+ 326us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:40.361523 (+ 334us) server_negotiation.cc:299] Negotiation successful
0810 14:47:40.361754 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 14:47:40.362807 9064 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:40.451140 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 8919
I20260810 14:47:40.472510 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 8470
I20260810 14:47:40.473919 8546 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:40.586256 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 8470
I20260810 14:47:40.615279 25438 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260810 14:47:40.959865 25438 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260810 14:47:40.960124 25438 mini_ranger.cc:66] Stopping Ranger...
I20260810 14:47:41.305532 25438 mini_ranger.cc:68] Stopped Ranger
2026-08-10 14:47:41.305 UTC [7321] LOG: received smart shutdown request
2026-08-10 14:47:41.309 UTC [7321] LOG: background worker "logical replication launcher" (PID 7331) exited with exit code 1
2026-08-10 14:47:41.309 UTC [7325] LOG: shutting down
2026-08-10 14:47:41.489 UTC [7321] LOG: database system is shut down
2026-08-10T14:47:41Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (54993 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:47:41 dist-test-slave-vkjp krb5kdc[9150](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:47:41 dist-test-slave-vkjp krb5kdc[9150](info): set up 2 sockets
Aug 10 14:47:41 dist-test-slave-vkjp krb5kdc[9150](info): commencing operation
krb5kdc: starting...
W20260810 14:47:42.827652 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.004s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:47:42 dist-test-slave-vkjp krb5kdc[9150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:47:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:47:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:47:43.095000 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:45293
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:45293,127.24.215.189:45317,127.24.215.188:40625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:47:43.434628 9164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:43.434908 9164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:43.435142 9164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:43.442270 9164 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:47:43.442370 9164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:43.442483 9164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:43.442584 9164 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:47:43.442665 9164 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:47:43.452508 9164 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:45293,127.24.215.189:45317,127.24.215.188:40625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:45293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:43.453835 9164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:43.455196 9164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:43.467607 9169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:43.467654 9172 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:43.467558 9170 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:43.468736 9164 server_base.cc:1034] running on GCE node
I20260810 14:47:43.469619 9164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:43.471020 9164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:43.472291 9164 hybrid_clock.cc:648] HybridClock initialized: now 1786373263472223 us; error 43 us; skew 500 ppm
I20260810 14:47:43.472653 9164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:43 dist-test-slave-vkjp krb5kdc[9150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:43.483644 9164 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:47:43.485172 9164 webserver.cc:460] Webserver started at http://127.24.215.190:41389/ using document root <none> and password file <none>
I20260810 14:47:43.485898 9164 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:43.486049 9164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:43.486367 9164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:47:43.489466 9164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "ee0f949367dd4b22a5753d5066d81c99"
format_stamp: "Formatted at 2026-08-10 14:47:43 on dist-test-slave-vkjp"
server_key: "401a9dcb1048bb333325b36690bd38d6"
server_key_iv: "fbee00afe1a9a011a9684d5f3c9f543b"
server_key_version: "encryptionkey@0"
I20260810 14:47:43.490276 9164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "ee0f949367dd4b22a5753d5066d81c99"
format_stamp: "Formatted at 2026-08-10 14:47:43 on dist-test-slave-vkjp"
server_key: "401a9dcb1048bb333325b36690bd38d6"
server_key_iv: "fbee00afe1a9a011a9684d5f3c9f543b"
server_key_version: "encryptionkey@0"
I20260810 14:47:43.495482 9164 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260810 14:47:43.498977 9179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:43.500351 9164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:47:43.500555 9164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "ee0f949367dd4b22a5753d5066d81c99"
format_stamp: "Formatted at 2026-08-10 14:47:43 on dist-test-slave-vkjp"
server_key: "401a9dcb1048bb333325b36690bd38d6"
server_key_iv: "fbee00afe1a9a011a9684d5f3c9f543b"
server_key_version: "encryptionkey@0"
I20260810 14:47:43.500759 9164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:43.519032 9164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:43.523208 9164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:43.523507 9164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:43.542443 9164 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:45293
I20260810 14:47:43.542479 9240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:45293 every 8 connection(s)
I20260810 14:47:43.543949 9164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:47:43.547618 9241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:47:43.549553 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 9164
I20260810 14:47:43.549839 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:47:43.550683 25438 external_mini_cluster.cc:1428] Setting key 6a30b7e13a629119190f994cba9712fc
I20260810 14:47:43.557327 9241 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:47:43 dist-test-slave-vkjp krb5kdc[9150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.215.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.189@KRBTEST.COM" created.
I20260810 14:47:43.583526 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:43.561320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:45293 (local address 127.0.0.1:59762)
0810 14:47:43.562677 (+ 1357us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:43.562734 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:43.565564 (+ 2830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:43.566461 (+ 897us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:43.566471 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:43.567395 (+ 924us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:43.567903 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:43.567929 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:43.569490 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:43.569498 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:43.571483 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:43.571501 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:43.572436 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:43.572445 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:43.572688 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:43.573433 (+ 745us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:43.573468 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:43.576207 (+ 2739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:43.579674 (+ 3467us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:43.579685 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:43.579698 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:43.580119 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:43.580530 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:43.580536 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:43.580539 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:43.580699 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:43.581575 (+ 876us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:43.581582 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:43.581783 (+ 201us) client_negotiation.cc:769] Sending connection context
0810 14:47:43.581997 (+ 214us) client_negotiation.cc:240] Negotiation successful
0810 14:47:43.582382 (+ 385us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1098,"spinlock_wait_cycles":49536,"thread_start_us":841,"threads_started":1}
I20260810 14:47:43.583564 9245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:43.561320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59762 (local address 127.24.215.190:45293)
0810 14:47:43.562295 (+ 975us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:43.562311 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:43.562861 (+ 550us) server_negotiation.cc:407] Connection header received
0810 14:47:43.565979 (+ 3118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:43.565985 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:43.566210 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:43.566368 (+ 158us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:43.568118 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:43.569315 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:43.571630 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:43.572182 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:43.572310 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:43.576440 (+ 4130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:43.576448 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:43.576460 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:43.576492 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:43.579511 (+ 3019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:43.580258 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:43.580272 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:43.580274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:43.580419 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:43.580814 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:43.580820 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:43.580828 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:43.581401 (+ 573us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:43.581638 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:43.582200 (+ 562us) server_negotiation.cc:299] Negotiation successful
0810 14:47:43.582594 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":691,"thread_start_us":424,"threads_started":1}
I20260810 14:47:43.586644 9241 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:47:43.588215 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:43.587642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:45317 (local address 127.0.0.1:60754)
0810 14:47:43.587920 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:43.588061 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260810 14:47:43.588544 9182 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.215.189:45317: connect: Connection refused (error 111)
I20260810 14:47:43.590720 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:43.590229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:45317 (local address 127.0.0.1:60768)
0810 14:47:43.590450 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:43.590537 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":107}
W20260810 14:47:43.591008 9241 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.189:45317: Network error: Client connection negotiation failed: client connection to 127.24.215.189:45317: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.189.keytab.
Entry for principal kudu/127.24.215.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.189.keytab.
I20260810 14:47:43.615234 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.189:45317
--webserver_interface=127.24.215.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.189.keytab
--principal=kudu/127.24.215.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:45293,127.24.215.189:45317,127.24.215.188:40625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:47:43.640551 9241 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } attempt: 1
I20260810 14:47:43.641767 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:43.641303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:45317 (local address 127.0.0.1:60772)
0810 14:47:43.641507 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:43.641638 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 14:47:43.642802 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:43.642463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:45317 (local address 127.0.0.1:60784)
0810 14:47:43.642602 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:43.642673 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 14:47:43.643111 9241 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.189:45317: Network error: Client connection negotiation failed: client connection to 127.24.215.189:45317: connect: Connection refused (error 111)
I20260810 14:47:43.711388 9241 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } attempt: 2
I20260810 14:47:43.712664 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:43.712166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:45317 (local address 127.0.0.1:60788)
0810 14:47:43.712428 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:43.712527 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 14:47:43.713742 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:43.713369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:45317 (local address 127.0.0.1:60800)
0810 14:47:43.713530 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:43.713614 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 14:47:43.713967 9241 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.189:45317: Network error: Client connection negotiation failed: client connection to 127.24.215.189:45317: connect: Connection refused (error 111)
I20260810 14:47:43.805281 9241 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } attempt: 3
I20260810 14:47:43.806414 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:43.805975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:45317 (local address 127.0.0.1:60804)
0810 14:47:43.806181 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:43.806275 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 14:47:43.807487 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:43.807121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:45317 (local address 127.0.0.1:60814)
0810 14:47:43.807280 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:43.807358 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 14:47:43.807682 9241 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.189:45317: Network error: Client connection negotiation failed: client connection to 127.24.215.189:45317: connect: Connection refused (error 111)
I20260810 14:47:43.950959 9241 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } attempt: 4
I20260810 14:47:43.952227 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:43.951688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:45317 (local address 127.0.0.1:60818)
0810 14:47:43.951996 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:43.952094 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260810 14:47:43.953255 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:43.952899 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:45317 (local address 127.0.0.1:60824)
0810 14:47:43.953044 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:43.953139 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:45317: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 14:47:43.953467 9241 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.189:45317: Network error: Client connection negotiation failed: client connection to 127.24.215.189:45317: connect: Connection refused (error 111)
W20260810 14:47:43.954306 9249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:43.954582 9249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:43.954777 9249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:43.962090 9249 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:47:43.962190 9249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:43.962304 9249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:43.962416 9249 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:47:43.962497 9249 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:47:43.972548 9249 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:45293,127.24.215.189:45317,127.24.215.188:40625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.189.keytab
--principal=kudu/127.24.215.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.189:45317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.215.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:43.973913 9249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:43.975221 9249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:43.987217 9255 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:43.987509 9258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:43.987514 9256 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:43.988901 9249 server_base.cc:1034] running on GCE node
I20260810 14:47:43.989708 9249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:43.991204 9249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:43.992437 9249 hybrid_clock.cc:648] HybridClock initialized: now 1786373263992387 us; error 57 us; skew 500 ppm
I20260810 14:47:43.992789 9249 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:43 dist-test-slave-vkjp krb5kdc[9150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:43.995903 9249 init.cc:377] Logged in from keytab as kudu/127.24.215.189@KRBTEST.COM (short username kudu)
I20260810 14:47:43.997385 9249 webserver.cc:460] Webserver started at http://127.24.215.189:43403/ using document root <none> and password file <none>
I20260810 14:47:43.998100 9249 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:43.998267 9249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:43.998592 9249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:47:44.001809 9249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/data/instance:
uuid: "9963427dbfb6402f9d3290ffcf600d9f"
format_stamp: "Formatted at 2026-08-10 14:47:43 on dist-test-slave-vkjp"
server_key: "bffc516c0f8d7f4467538d2b30fca113"
server_key_iv: "0c8cb0c40c7c68e9a1334d39212f760b"
server_key_version: "encryptionkey@0"
I20260810 14:47:44.002612 9249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/wal/instance:
uuid: "9963427dbfb6402f9d3290ffcf600d9f"
format_stamp: "Formatted at 2026-08-10 14:47:43 on dist-test-slave-vkjp"
server_key: "bffc516c0f8d7f4467538d2b30fca113"
server_key_iv: "0c8cb0c40c7c68e9a1334d39212f760b"
server_key_version: "encryptionkey@0"
I20260810 14:47:44.008035 9249 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 14:47:44.011590 9266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:44.013018 9249 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 14:47:44.013168 9249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/wal
uuid: "9963427dbfb6402f9d3290ffcf600d9f"
format_stamp: "Formatted at 2026-08-10 14:47:43 on dist-test-slave-vkjp"
server_key: "bffc516c0f8d7f4467538d2b30fca113"
server_key_iv: "0c8cb0c40c7c68e9a1334d39212f760b"
server_key_version: "encryptionkey@0"
I20260810 14:47:44.013348 9249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:44.029215 9249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:44.033273 9249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:44.033519 9249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:44.052316 9249 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.189:45317
I20260810 14:47:44.052352 9324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.189:45317 every 8 connection(s)
I20260810 14:47:44.053710 9249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/data/info.pb
I20260810 14:47:44.057314 9325 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:47:44.060458 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 9249
I20260810 14:47:44.060658 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-1/wal/instance
I20260810 14:47:44.061113 25438 external_mini_cluster.cc:1428] Setting key 95d67b4625a7556e4d79a7011ad68b39
I20260810 14:47:44.063937 9325 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:47:44 dist-test-slave-vkjp krb5kdc[9150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.189@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.24.215.188@KRBTEST.COM; defaulting to no policy
I20260810 14:47:44.087153 9245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.066583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59776 (local address 127.24.215.190:45293)
0810 14:47:44.066812 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:44.066819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:44.068051 (+ 1232us) server_negotiation.cc:407] Connection header received
0810 14:47:44.071023 (+ 2972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:44.071036 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:44.071154 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:44.071277 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:44.072711 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.073887 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.075482 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.076287 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.076434 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.080424 (+ 3990us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:44.080428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:44.080432 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:44.080473 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:44.083451 (+ 2978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.084538 (+ 1087us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.084542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.084544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.084666 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.085177 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.085180 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.085182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.085468 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:44.085577 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:44.086635 (+ 1058us) server_negotiation.cc:299] Negotiation successful
0810 14:47:44.086809 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 14:47:44.088344 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.066995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:45293 (local address 127.0.0.1:59776)
0810 14:47:44.067860 (+ 865us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.067910 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:44.070703 (+ 2793us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:44.071471 (+ 768us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:44.071478 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:44.071967 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:44.072505 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:44.072521 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.074052 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.074055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:44.075333 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:44.075346 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.076496 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.076501 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:44.076686 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.077392 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:44.077415 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:44.080202 (+ 2787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:44.083611 (+ 3409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:44.083625 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:44.083647 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:44.084413 (+ 766us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:44.084780 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:44.084785 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:44.084788 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:44.085057 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:44.085618 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:44.085631 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:44.086199 (+ 568us) client_negotiation.cc:769] Sending connection context
0810 14:47:44.086542 (+ 343us) client_negotiation.cc:240] Negotiation successful
0810 14:47:44.087150 (+ 608us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":588,"thread_start_us":327,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.188@KRBTEST.COM" created.
I20260810 14:47:44.090221 9325 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:47:44 dist-test-slave-vkjp krb5kdc[9150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.189@KRBTEST.COM for kudu/127.24.215.189@KRBTEST.COM
I20260810 14:47:44.112072 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.091153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:45317 (local address 127.0.0.1:60832)
0810 14:47:44.091350 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.091376 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:44.091524 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:44.096295 (+ 4771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:44.096299 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:44.096734 (+ 435us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:44.096883 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:44.096892 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.098921 (+ 2029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.098925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:44.099859 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:44.099871 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.101122 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.101130 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:44.101414 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.102056 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:44.102085 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:44.104787 (+ 2702us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:44.109429 (+ 4642us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:44.109433 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:44.109436 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:44.109799 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:44.110311 (+ 512us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:44.110314 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:44.110316 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:44.110455 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:44.111341 (+ 886us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:44.111344 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:44.111479 (+ 135us) client_negotiation.cc:769] Sending connection context
0810 14:47:44.111592 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 14:47:44.111803 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260810 14:47:44.112668 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.091173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60832 (local address 127.24.215.189:45317)
0810 14:47:44.095694 (+ 4521us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:44.095705 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:44.095744 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:47:44.095809 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:44.095820 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:44.096092 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:44.096276 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:44.097052 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.098757 (+ 1705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.100017 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.100931 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.101053 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.105059 (+ 4006us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:44.105077 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:44.105099 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:44.105143 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:44.109277 (+ 4134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.109960 (+ 683us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.109971 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.109977 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.110150 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.110570 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.110577 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.110583 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.111211 (+ 628us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:44.111457 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:44.111802 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 14:47:44.112125 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4310,"thread_start_us":4104,"threads_started":1}
I20260810 14:47:44.115275 9325 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:47:44.116490 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.116047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58430)
0810 14:47:44.116217 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.116335 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260810 14:47:44.116760 9268 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
I20260810 14:47:44.118479 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.118138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58442)
0810 14:47:44.118313 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.118382 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 14:47:44.118705 9325 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:40625: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.188.keytab.
Entry for principal kudu/127.24.215.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.188.keytab.
I20260810 14:47:44.126446 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.188:40625
--webserver_interface=127.24.215.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.188.keytab
--principal=kudu/127.24.215.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45915
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:45293,127.24.215.189:45317,127.24.215.188:40625
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:47:44.168062 9325 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } attempt: 1
I20260810 14:47:44.169410 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.168846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58444)
0810 14:47:44.169059 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.169217 (+ 158us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260810 14:47:44.170593 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.170208 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58450)
0810 14:47:44.170412 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.170486 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260810 14:47:44.170864 9325 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:40625: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
I20260810 14:47:44.239143 9325 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } attempt: 2
I20260810 14:47:44.240386 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.239944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58458)
0810 14:47:44.240140 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.240230 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 14:47:44.241401 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.241068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58474)
0810 14:47:44.241230 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.241297 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 14:47:44.241598 9325 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:40625: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
I20260810 14:47:44.252827 9241 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } attempt: 5
Aug 10 14:47:44 dist-test-slave-vkjp krb5kdc[9150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for kudu/127.24.215.189@KRBTEST.COM
I20260810 14:47:44.267782 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.253490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:45317 (local address 127.0.0.1:60838)
0810 14:47:44.253675 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.253700 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:44.253803 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:44.254206 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:44.254209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:44.254571 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:44.254811 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:44.254824 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.256222 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.256226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:44.257078 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:44.257088 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.258040 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.258044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:44.258181 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.258811 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:44.258828 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:44.261074 (+ 2246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:44.265408 (+ 4334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:44.265412 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:44.265415 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:44.265787 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:44.266213 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:44.266217 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:44.266220 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:44.266346 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:44.266986 (+ 640us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:44.266990 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:44.267178 (+ 188us) client_negotiation.cc:769] Sending connection context
0810 14:47:44.267305 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 14:47:44.267515 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 14:47:44.268013 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.253580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60838 (local address 127.24.215.189:45317)
0810 14:47:44.253798 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:44.253806 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:44.253845 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:47:44.253924 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:44.253932 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:44.254048 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:44.254183 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:44.254984 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.256090 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.257219 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.257903 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.258034 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.261280 (+ 3246us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:44.261290 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:44.261297 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:44.261345 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:44.265224 (+ 3879us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.265915 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.265921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.265924 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.266097 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.266452 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.266457 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.266460 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.266861 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:44.267004 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:44.267444 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 14:47:44.267693 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 14:47:44.269052 9241 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:47:44.270051 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.269683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58476)
0810 14:47:44.269841 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.269926 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 14:47:44.270962 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.270636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58486)
0810 14:47:44.270772 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.270858 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 14:47:44.271211 9241 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:40625: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
I20260810 14:47:44.322427 9241 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } attempt: 1
I20260810 14:47:44.323724 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.323169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58490)
0810 14:47:44.323398 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.323513 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 14:47:44.324783 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.324381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58496)
0810 14:47:44.324535 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.324633 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 14:47:44.325028 9241 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:40625: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
I20260810 14:47:44.332808 9325 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } attempt: 3
I20260810 14:47:44.334038 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.333392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58506)
0810 14:47:44.333661 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.333799 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 14:47:44.335340 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.334733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58508)
0810 14:47:44.335033 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.335171 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":166}
W20260810 14:47:44.335626 9325 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:40625: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
I20260810 14:47:44.393285 9241 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } attempt: 2
I20260810 14:47:44.394636 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.394014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58522)
0810 14:47:44.394286 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.394411 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 14:47:44.395797 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.395394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58530)
0810 14:47:44.395554 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.395653 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 14:47:44.396095 9241 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:40625: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
W20260810 14:47:44.474777 9332 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:44.475061 9332 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:44.475258 9332 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260810 14:47:44.478875 9325 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } attempt: 4
I20260810 14:47:44.480140 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.479546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58542)
0810 14:47:44.479812 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.479935 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260810 14:47:44.481261 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.480877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58546)
0810 14:47:44.481043 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.481128 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 14:47:44.481479 9325 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:40625: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
W20260810 14:47:44.483450 9332 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:47:44.483573 9332 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:44.483681 9332 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:44.483783 9332 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:47:44.483865 9332 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:47:44.494143 9332 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45915
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:45293,127.24.215.189:45317,127.24.215.188:40625
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.188.keytab
--principal=kudu/127.24.215.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.188:40625
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.215.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:44.495667 9332 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:44.497002 9332 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 14:47:44.502381 9241 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } attempt: 3
I20260810 14:47:44.503896 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.503191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58554)
0810 14:47:44.503482 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.503628 (+ 146us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 14:47:44.505216 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.504639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58568)
0810 14:47:44.504948 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.505070 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162}
W20260810 14:47:44.505448 9241 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:40625: Network error: Client connection negotiation failed: client connection to 127.24.215.188:40625: connect: Connection refused (error 111)
W20260810 14:47:44.511150 9339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:44.511179 9342 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:44.511449 9338 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:44.512277 9332 server_base.cc:1034] running on GCE node
I20260810 14:47:44.513065 9332 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:44.514452 9332 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:44.515724 9332 hybrid_clock.cc:648] HybridClock initialized: now 1786373264515653 us; error 51 us; skew 500 ppm
I20260810 14:47:44.516165 9332 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:44 dist-test-slave-vkjp krb5kdc[9150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373264, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:44.519685 9332 init.cc:377] Logged in from keytab as kudu/127.24.215.188@KRBTEST.COM (short username kudu)
I20260810 14:47:44.521363 9332 webserver.cc:460] Webserver started at http://127.24.215.188:34537/ using document root <none> and password file <none>
I20260810 14:47:44.522066 9332 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:44.522184 9332 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:44.522473 9332 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:47:44.525748 9332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/data/instance:
uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1"
format_stamp: "Formatted at 2026-08-10 14:47:44 on dist-test-slave-vkjp"
server_key: "c1a982f3c85dbecf030b4af9bf104a73"
server_key_iv: "ee14a699ab4e4ae30c7bff132c4b87b2"
server_key_version: "encryptionkey@0"
I20260810 14:47:44.526607 9332 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/wal/instance:
uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1"
format_stamp: "Formatted at 2026-08-10 14:47:44 on dist-test-slave-vkjp"
server_key: "c1a982f3c85dbecf030b4af9bf104a73"
server_key_iv: "ee14a699ab4e4ae30c7bff132c4b87b2"
server_key_version: "encryptionkey@0"
I20260810 14:47:44.532234 9332 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 14:47:44.535820 9349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:44.537173 9332 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260810 14:47:44.537381 9332 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/wal
uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1"
format_stamp: "Formatted at 2026-08-10 14:47:44 on dist-test-slave-vkjp"
server_key: "c1a982f3c85dbecf030b4af9bf104a73"
server_key_iv: "ee14a699ab4e4ae30c7bff132c4b87b2"
server_key_version: "encryptionkey@0"
I20260810 14:47:44.537585 9332 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:44.557298 9332 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:44.561354 9332 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:44.561635 9332 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:44.581808 9332 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.188:40625
I20260810 14:47:44.581841 9407 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.188:40625 every 8 connection(s)
I20260810 14:47:44.583343 9332 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/data/info.pb
I20260810 14:47:44.587050 9408 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:47:44.592705 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 9332
I20260810 14:47:44.592935 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786373051600012-25438-0/minicluster-data/master-2/wal/instance
I20260810 14:47:44.593398 25438 external_mini_cluster.cc:1428] Setting key eb83a8d9e27794e5292160d3953a6059
I20260810 14:47:44.593611 9408 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:47:44 dist-test-slave-vkjp krb5kdc[9150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373264, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.188@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
Aug 10 14:47:44 dist-test-slave-vkjp krb5kdc[9150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.188@KRBTEST.COM
I20260810 14:47:44.613581 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.596620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59790 (local address 127.24.215.190:45293)
0810 14:47:44.597187 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:44.597195 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:44.597734 (+ 539us) server_negotiation.cc:407] Connection header received
0810 14:47:44.598842 (+ 1108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:44.598847 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:44.598929 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:44.599059 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:44.600347 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.601134 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.602611 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.603036 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.603219 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.607260 (+ 4041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:44.607265 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:44.607268 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:44.607302 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:44.610295 (+ 2993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.611163 (+ 868us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.611167 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.611169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.611297 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.611753 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.611757 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.611760 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.612026 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:44.612200 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:44.612914 (+ 714us) server_negotiation.cc:299] Negotiation successful
0810 14:47:44.613150 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":310,"threads_started":1}
I20260810 14:47:44.614600 9413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.596685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:45293 (local address 127.0.0.1:59790)
0810 14:47:44.597577 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.597614 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:44.598598 (+ 984us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:44.599168 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:44.599175 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:44.599787 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:44.600168 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:44.600185 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.601304 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.601308 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:44.602459 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:44.602470 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.603328 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.603336 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:44.603611 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.604352 (+ 741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:44.604374 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:44.607015 (+ 2641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:44.610457 (+ 3442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:44.610469 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:44.610481 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:44.610961 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:44.611417 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:44.611422 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:44.611426 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:44.611622 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:44.612172 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:44.612182 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:44.612533 (+ 351us) client_negotiation.cc:769] Sending connection context
0810 14:47:44.612817 (+ 284us) client_negotiation.cc:240] Negotiation successful
0810 14:47:44.613314 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"mutex_wait_us":48,"thread_start_us":507,"threads_started":1}
I20260810 14:47:44.614619 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.595315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58578 (local address 127.24.215.188:40625)
0810 14:47:44.596174 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:44.596187 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:44.596234 (+ 47us) server_negotiation.cc:407] Connection header received
0810 14:47:44.598586 (+ 2352us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:44.598607 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:44.598843 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:44.599059 (+ 216us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:44.600636 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.601829 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.603216 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.603798 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.603936 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.608553 (+ 4617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:44.608563 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:44.608576 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:44.608605 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:44.611378 (+ 2773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.612087 (+ 709us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.612102 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.612104 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.612228 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.612632 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.612636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.612638 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.613019 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:44.613332 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:44.613678 (+ 346us) server_negotiation.cc:299] Negotiation successful
0810 14:47:44.613943 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":607,"spinlock_wait_cycles":15616,"thread_start_us":367,"threads_started":1}
I20260810 14:47:44.616364 9408 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:47:44 dist-test-slave-vkjp krb5kdc[9150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373264, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.188@KRBTEST.COM for kudu/127.24.215.189@KRBTEST.COM
Aug 10 14:47:44 dist-test-slave-vkjp krb5kdc[9150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:47:44.629485 9413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.617090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:45317 (local address 127.0.0.1:60842)
0810 14:47:44.617257 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.617277 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:44.617374 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:44.617689 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:44.617694 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:44.618136 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:44.618333 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:44.618345 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.619365 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.619371 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:44.620255 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:44.620266 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.620966 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.620972 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:44.621216 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.621784 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:44.621800 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:44.623759 (+ 1959us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:44.626963 (+ 3204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:44.626968 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:44.626972 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:44.627423 (+ 451us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:44.627810 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:44.627815 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:44.627819 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:44.627975 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:44.628490 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:44.628495 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:44.628657 (+ 162us) client_negotiation.cc:769] Sending connection context
0810 14:47:44.628799 (+ 142us) client_negotiation.cc:240] Negotiation successful
0810 14:47:44.629103 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260810 14:47:44.629500 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.617188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60842 (local address 127.24.215.189:45317)
0810 14:47:44.617378 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:44.617384 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:44.617405 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:47:44.617470 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:44.617478 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:44.617552 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:44.617658 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:44.618479 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.619230 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.620398 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.620824 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.620966 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.623960 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:44.623965 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:44.623968 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:44.624001 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:44.626813 (+ 2812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.627548 (+ 735us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.627555 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.627559 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.627696 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.628085 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.628091 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.628095 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.628363 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:44.628522 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:44.628890 (+ 368us) server_negotiation.cc:299] Negotiation successful
0810 14:47:44.629119 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:47:44.630471 9408 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:47:44.633123 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.618177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59792 (local address 127.24.215.190:45293)
0810 14:47:44.618361 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:44.618366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:44.618501 (+ 135us) server_negotiation.cc:407] Connection header received
0810 14:47:44.618684 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:44.618689 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:44.618758 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:44.618865 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:44.620165 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.620832 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.622257 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.622661 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.622797 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.627261 (+ 4464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:44.627266 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:44.627269 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:44.627301 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:44.630159 (+ 2858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.630982 (+ 823us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.630987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.630990 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.631198 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.631636 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.631642 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.631645 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.631963 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:44.632242 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:44.632567 (+ 325us) server_negotiation.cc:299] Negotiation successful
0810 14:47:44.632804 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
Aug 10 14:47:44 dist-test-slave-vkjp krb5kdc[9150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373264, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.188@KRBTEST.COM for kudu/127.24.215.188@KRBTEST.COM
Aug 10 14:47:44 dist-test-slave-vkjp krb5kdc[9150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373262, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.189@KRBTEST.COM
I20260810 14:47:44.644238 9413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.631273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58582)
0810 14:47:44.631467 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.631488 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:44.631600 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:44.631869 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:44.631873 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:44.632246 (+ 373us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:44.632395 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:44.632402 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.633526 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.633533 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:44.634596 (+ 1063us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:44.634608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.635515 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.635521 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:44.635749 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.636312 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:44.636338 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:44.638706 (+ 2368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:44.641847 (+ 3141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:44.641853 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:44.641856 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:44.642197 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:44.642573 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:44.642579 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:44.642582 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:44.642757 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:44.643394 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:44.643399 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:44.643541 (+ 142us) client_negotiation.cc:769] Sending connection context
0810 14:47:44.643663 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 14:47:44.643975 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64}
I20260810 14:47:44.644757 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.631456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58582 (local address 127.24.215.188:40625)
0810 14:47:44.631606 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:44.631610 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:44.631628 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:47:44.631701 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:44.631706 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:44.631762 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:44.631856 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:44.632536 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.633387 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.634732 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.635369 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.635486 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.638888 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:44.638894 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:44.638909 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:44.638936 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:44.641726 (+ 2790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.642311 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.642319 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.642325 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.642465 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.642879 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.642884 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.642887 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.643271 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:44.643435 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:44.643801 (+ 366us) server_negotiation.cc:299] Negotiation successful
0810 14:47:44.644174 (+ 373us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":104,"server-negotiator.queue_time_us":55}
I20260810 14:47:44.650342 9408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1: Bootstrap starting.
I20260810 14:47:44.650635 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.635002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60854 (local address 127.24.215.189:45317)
0810 14:47:44.635199 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:44.635204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:44.635224 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:47:44.635272 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:44.635277 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:44.635351 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:44.635464 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:44.637073 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.637970 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.639337 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.639976 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.640129 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.644982 (+ 4853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:44.644989 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:44.644994 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:44.645032 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:44.647885 (+ 2853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.648698 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.648702 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.648704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.648862 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.649302 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.649306 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.649308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.649564 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:44.649776 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:44.650146 (+ 370us) server_negotiation.cc:299] Negotiation successful
0810 14:47:44.650320 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:47:44.655046 9408 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1: Neither blocks nor log segments found. Creating new log.
I20260810 14:47:44.655874 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:47:44.656390 9408 log.cc:824] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1: Log is configured to *not* fsync() on all Append() calls
I20260810 14:47:44.659353 9408 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1: No bootstrap required, opened a new log
I20260810 14:47:44.674527 9408 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } }
I20260810 14:47:44.675804 9408 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:47:44.676003 9408 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c1cc325ea0147d4b6bffd05ea1da3c1, State: Initialized, Role: FOLLOWER
I20260810 14:47:44.676786 9408 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } }
I20260810 14:47:44.682669 9241 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } attempt: 4
I20260810 14:47:44.682770 9424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } } }
I20260810 14:47:44.684373 9408 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:47:44.684490 9424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:47:44.684523 9412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.661183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59800 (local address 127.24.215.190:45293)
0810 14:47:44.661405 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:44.661414 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:44.661968 (+ 554us) server_negotiation.cc:407] Connection header received
0810 14:47:44.662261 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:44.662269 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:44.662361 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:44.662455 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:44.665563 (+ 3108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.667058 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.670085 (+ 3027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.671008 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.671167 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.675997 (+ 4830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:44.676005 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:44.676013 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:44.676061 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:44.679873 (+ 3812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.680846 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.680853 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.680858 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.681014 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.681556 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.681564 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.681567 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.681947 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:44.683237 (+ 1290us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:44.683831 (+ 594us) server_negotiation.cc:299] Negotiation successful
0810 14:47:44.684082 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
Aug 10 14:47:44 dist-test-slave-vkjp krb5kdc[9150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for kudu/127.24.215.188@KRBTEST.COM
I20260810 14:47:44.698457 9329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.661647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60866 (local address 127.24.215.189:45317)
0810 14:47:44.661831 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:44.661836 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:44.674084 (+ 12248us) server_negotiation.cc:407] Connection header received
0810 14:47:44.674639 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:44.674649 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:44.674743 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:44.674830 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:44.676882 (+ 2052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.678071 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.680478 (+ 2407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.681220 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.681322 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.688319 (+ 6997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:44.688327 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:44.688332 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:44.688375 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:44.694493 (+ 6118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.695217 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.695221 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.695223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.695424 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.695918 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.695923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.695926 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.696345 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:44.696615 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:44.697605 (+ 990us) server_negotiation.cc:299] Negotiation successful
0810 14:47:44.697912 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 14:47:44.701165 9411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.662455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58584 (local address 127.24.215.188:40625)
0810 14:47:44.662648 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:44.662653 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:44.675904 (+ 13251us) server_negotiation.cc:407] Connection header received
0810 14:47:44.676299 (+ 395us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:44.676304 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:44.676383 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:44.676455 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:44.678968 (+ 2513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.680453 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.682971 (+ 2518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.683900 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.684029 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.689007 (+ 4978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:44.689016 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:44.689023 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:44.689075 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:44.696712 (+ 7637us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.697719 (+ 1007us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.697726 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.697729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.697979 (+ 250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.698509 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.698515 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.698518 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.698986 (+ 468us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:44.699252 (+ 266us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:44.700506 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0810 14:47:44.700727 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 14:47:44.703455 9246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.683299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58586)
0810 14:47:44.683716 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.683737 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:44.683821 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:44.686146 (+ 2325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:44.686153 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:44.686639 (+ 486us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:44.686876 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:44.686903 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.688793 (+ 1890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.688801 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:44.690411 (+ 1610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:44.690427 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.691530 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.691536 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:44.691709 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.692315 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:44.692331 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:44.694455 (+ 2124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:44.700069 (+ 5614us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:44.700075 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:44.700081 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:44.700626 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:44.701109 (+ 483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:44.701115 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:44.701119 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:44.701310 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:44.702657 (+ 1347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:44.702661 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:44.702807 (+ 146us) client_negotiation.cc:769] Sending connection context
0810 14:47:44.702916 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 14:47:44.703184 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274}
I20260810 14:47:44.704414 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.684026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58586 (local address 127.24.215.188:40625)
0810 14:47:44.685768 (+ 1742us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:44.685774 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:44.685799 (+ 25us) server_negotiation.cc:407] Connection header received
0810 14:47:44.685850 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:44.685857 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:44.685949 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:44.687054 (+ 1105us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:44.687404 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.688636 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.690604 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.691338 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.692030 (+ 692us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.695411 (+ 3381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:44.695422 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:44.695439 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:44.695492 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:44.699876 (+ 4384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.700791 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.700795 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.700798 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.700948 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.701939 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.701943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.701947 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.702374 (+ 427us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:44.703821 (+ 1447us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:44.703971 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 14:47:44.704119 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1587,"thread_start_us":1534,"threads_started":1}
I20260810 14:47:44.709867 9241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99: Bootstrap starting.
W20260810 14:47:44.712878 9438 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260810 14:47:44.713645 9241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99: Neither blocks nor log segments found. Creating new log.
W20260810 14:47:44.713860 9438 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:47:44.714761 9241 log.cc:824] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99: Log is configured to *not* fsync() on all Append() calls
I20260810 14:47:44.718411 9241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99: No bootstrap required, opened a new log
I20260810 14:47:44.713029 9332 master_runner.cc:186] Error getting master registration for 127.24.215.190:45293: OK, instance_id { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" instance_seqno: 1786373263536907 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:47:44.725366 9332 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 14:47:44.727151 9241 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } }
I20260810 14:47:44.727489 9241 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:47:44.727603 9241 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee0f949367dd4b22a5753d5066d81c99, State: Initialized, Role: FOLLOWER
I20260810 14:47:44.728195 9241 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } }
I20260810 14:47:44.729130 9441 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } } }
I20260810 14:47:44.729568 9441 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:47:44.730163 9241 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99 [sys.catalog]: configured and running, proceeding with master startup.
W20260810 14:47:44.743053 9453 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:47:44.743294 9453 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:47:44.741243 9164 master_runner.cc:186] Error getting master registration for 127.24.215.189:45317: OK, instance_id { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" instance_seqno: 1786373264046874 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:47:44.749629 9164 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 14:47:44.776567 9186 tablet.cc:2369] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:47:44.780701 9325 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } attempt: 5
Aug 10 14:47:44 dist-test-slave-vkjp krb5kdc[9150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373263, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.189@KRBTEST.COM for kudu/127.24.215.188@KRBTEST.COM
I20260810 14:47:44.798182 9327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.781363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:40625 (local address 127.0.0.1:58602)
0810 14:47:44.781649 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:44.781673 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:44.781816 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:44.782229 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:44.782235 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:44.782825 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:44.783034 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:44.783047 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.784547 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.784555 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:44.785997 (+ 1442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:44.786014 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.787024 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.787031 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:44.787294 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.788013 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:44.788038 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:44.790972 (+ 2934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:44.795317 (+ 4345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:44.795323 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:44.795327 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:44.795843 (+ 516us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:44.796307 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:44.796315 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:44.796320 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:44.796505 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:44.797108 (+ 603us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:44.797113 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:44.797308 (+ 195us) client_negotiation.cc:769] Sending connection context
0810 14:47:44.797490 (+ 182us) client_negotiation.cc:240] Negotiation successful
0810 14:47:44.797795 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260810 14:47:44.798218 9426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:44.781437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58602 (local address 127.24.215.188:40625)
0810 14:47:44.781617 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:44.781623 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:44.781763 (+ 140us) server_negotiation.cc:407] Connection header received
0810 14:47:44.781941 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:44.781947 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:44.782060 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:44.782162 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:44.783204 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.784391 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.786158 (+ 1767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:44.786835 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:44.786972 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:44.791232 (+ 4260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:44.791240 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:44.791245 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:44.791283 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:44.795163 (+ 3880us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.795978 (+ 815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.795984 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.795988 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.796177 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:44.796625 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:44.796630 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:44.796634 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:44.796971 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:44.797151 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:44.797601 (+ 450us) server_negotiation.cc:299] Negotiation successful
0810 14:47:44.797845 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 14:47:44.804943 9325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f: Bootstrap starting.
I20260810 14:47:44.811206 9325 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f: Neither blocks nor log segments found. Creating new log.
I20260810 14:47:44.812592 9325 log.cc:824] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f: Log is configured to *not* fsync() on all Append() calls
W20260810 14:47:44.813764 9355 tablet.cc:2369] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:47:44.816133 9325 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f: No bootstrap required, opened a new log
I20260810 14:47:44.828087 9325 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } }
I20260810 14:47:44.828563 9325 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:47:44.828699 9325 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9963427dbfb6402f9d3290ffcf600d9f, State: Initialized, Role: FOLLOWER
I20260810 14:47:44.829416 9325 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } }
I20260810 14:47:44.830511 9457 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } } }
I20260810 14:47:44.831070 9457 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:47:44.832175 9325 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [sys.catalog]: configured and running, proceeding with master startup.
W20260810 14:47:44.854411 9470 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260810 14:47:44.859829 9249 master_runner.cc:263] No leader master found from master 9963427dbfb6402f9d3290ffcf600d9f
I20260810 14:47:44.860034 9249 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 14:47:44.860208 9470 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:47:44.916663 9457 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:47:44.916987 9457 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } }
I20260810 14:47:44.918766 9457 leader_election.cc:288] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ee0f949367dd4b22a5753d5066d81c99 (127.24.215.190:45293), 1c1cc325ea0147d4b6bffd05ea1da3c1 (127.24.215.188:40625)
I20260810 14:47:44.920095 9213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9963427dbfb6402f9d3290ffcf600d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee0f949367dd4b22a5753d5066d81c99" is_pre_election: true
I20260810 14:47:44.920747 9213 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9963427dbfb6402f9d3290ffcf600d9f in term 0.
I20260810 14:47:44.921576 9269 leader_election.cc:302] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9963427dbfb6402f9d3290ffcf600d9f, ee0f949367dd4b22a5753d5066d81c99; no voters:
I20260810 14:47:44.922833 9380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9963427dbfb6402f9d3290ffcf600d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" is_pre_election: true
I20260810 14:47:44.923542 9380 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9963427dbfb6402f9d3290ffcf600d9f in term 0.
I20260810 14:47:44.924494 9457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:47:44.924646 9457 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:47:44.924738 9457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:47:44.928280 9457 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } }
I20260810 14:47:44.929529 9213 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9963427dbfb6402f9d3290ffcf600d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee0f949367dd4b22a5753d5066d81c99"
I20260810 14:47:44.929774 9213 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:47:44.945755 9213 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9963427dbfb6402f9d3290ffcf600d9f in term 1.
I20260810 14:47:44.950302 9269 leader_election.cc:302] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9963427dbfb6402f9d3290ffcf600d9f, ee0f949367dd4b22a5753d5066d81c99; no voters:
I20260810 14:47:44.952684 9380 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "9963427dbfb6402f9d3290ffcf600d9f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1"
I20260810 14:47:44.952958 9380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:47:44.956681 9380 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9963427dbfb6402f9d3290ffcf600d9f in term 1.
I20260810 14:47:44.957607 9457 leader_election.cc:288] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [CANDIDATE]: Term 1 election: Requested vote from peers ee0f949367dd4b22a5753d5066d81c99 (127.24.215.190:45293), 1c1cc325ea0147d4b6bffd05ea1da3c1 (127.24.215.188:40625)
I20260810 14:47:44.958094 9457 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:47:44.958511 9457 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [term 1 LEADER]: Becoming Leader. State: Replica: 9963427dbfb6402f9d3290ffcf600d9f, State: Running, Role: LEADER
I20260810 14:47:44.959589 9457 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } }
I20260810 14:47:44.981730 9472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9963427dbfb6402f9d3290ffcf600d9f. Latest consensus state: current_term: 1 leader_uuid: "9963427dbfb6402f9d3290ffcf600d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } } }
I20260810 14:47:44.982270 9472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [sys.catalog]: This master's current role is: LEADER
I20260810 14:47:44.995648 9474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:47:44.998062 9474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:47:45.020251 9380 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1 [term 1 FOLLOWER]: Refusing update from remote peer 9963427dbfb6402f9d3290ffcf600d9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:47:45.021559 9472 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:47:45.034286 9213 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99 [term 1 FOLLOWER]: Refusing update from remote peer 9963427dbfb6402f9d3290ffcf600d9f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:47:45.035507 9472 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260810 14:47:45.036106 9272 tablet.cc:2369] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:47:45.072283 9441 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9963427dbfb6402f9d3290ffcf600d9f. Latest consensus state: current_term: 1 leader_uuid: "9963427dbfb6402f9d3290ffcf600d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } } }
I20260810 14:47:45.072767 9441 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:47:45.075263 9475 mvcc.cc:205] Tried to move back new op lower bound from 7316984893511725056 to 7316984893287747584. Current Snapshot: MvccSnapshot[applied={T|T < 7316984893511725056}]
I20260810 14:47:45.076278 9477 mvcc.cc:205] Tried to move back new op lower bound from 7316984893511725056 to 7316984893287747584. Current Snapshot: MvccSnapshot[applied={T|T < 7316984893511725056}]
I20260810 14:47:45.082829 9424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9963427dbfb6402f9d3290ffcf600d9f. Latest consensus state: current_term: 1 leader_uuid: "9963427dbfb6402f9d3290ffcf600d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } } }
I20260810 14:47:45.083220 9424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:47:45.087191 9476 mvcc.cc:205] Tried to move back new op lower bound from 7316984893511725056 to 7316984893287747584. Current Snapshot: MvccSnapshot[applied={T|T < 7316984893511725056}]
I20260810 14:47:45.104849 9441 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9963427dbfb6402f9d3290ffcf600d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } } }
I20260810 14:47:45.105156 9441 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:47:45.105650 9424 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9963427dbfb6402f9d3290ffcf600d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } } }
I20260810 14:47:45.105928 9424 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:47:45.111388 9472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9963427dbfb6402f9d3290ffcf600d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } } }
I20260810 14:47:45.111714 9472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [sys.catalog]: This master's current role is: LEADER
I20260810 14:47:45.112121 9474 catalog_manager.cc:1347] Generated new cluster ID: 463c16c76fb04591858a29a57cdd270b
I20260810 14:47:45.112226 9474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:47:45.112443 9472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9963427dbfb6402f9d3290ffcf600d9f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ee0f949367dd4b22a5753d5066d81c99" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 45293 } } peers { permanent_uuid: "9963427dbfb6402f9d3290ffcf600d9f" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 45317 } } peers { permanent_uuid: "1c1cc325ea0147d4b6bffd05ea1da3c1" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 40625 } } }
I20260810 14:47:45.112727 9472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f [sys.catalog]: This master's current role is: LEADER
I20260810 14:47:45.141191 9474 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:47:45.144222 9474 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:47:45.171886 9474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9963427dbfb6402f9d3290ffcf600d9f: Generated new TSK 0
I20260810 14:47:45.173135 9474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:47:45.715979 9438 catalog_manager.cc:1259] Loaded cluster ID: 463c16c76fb04591858a29a57cdd270b
I20260810 14:47:45.716086 9438 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1: loading cluster ID for follower catalog manager: success
I20260810 14:47:45.718590 9438 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1: acquiring CA information for follower catalog manager: success
I20260810 14:47:45.720419 9438 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1c1cc325ea0147d4b6bffd05ea1da3c1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:47:45.745287 9453 catalog_manager.cc:1259] Loaded cluster ID: 463c16c76fb04591858a29a57cdd270b
I20260810 14:47:45.745400 9453 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99: loading cluster ID for follower catalog manager: success
I20260810 14:47:45.748028 9453 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99: acquiring CA information for follower catalog manager: success
I20260810 14:47:45.749810 9453 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ee0f949367dd4b22a5753d5066d81c99: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:47:45.806663 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 9164
I20260810 14:47:45.808068 9235 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:45.918419 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 9164
I20260810 14:47:45.944872 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 9249
I20260810 14:47:45.946089 9320 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:46.058140 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 9249
W20260810 14:47:46.082266 9353 connection.cc:535] server connection from 127.0.0.1:58602 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 14:47:46.082845 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 9332
I20260810 14:47:46.084082 9403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:46.191552 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 9332
2026-08-10T14:47:46Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4568 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:47:46 dist-test-slave-vkjp krb5kdc[9494](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:47:46 dist-test-slave-vkjp krb5kdc[9494](info): set up 2 sockets
Aug 10 14:47:46 dist-test-slave-vkjp krb5kdc[9494](info): commencing operation
krb5kdc: starting...
W20260810 14:47:47.392781 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:47:47 dist-test-slave-vkjp krb5kdc[9494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:47:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:47:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:47:47.666790 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:34141
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:37315
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:34141
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:47:48.009451 9512 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:48.009817 9512 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:48.009994 9512 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:48.017555 9512 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:47:48.017871 9512 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:48.018003 9512 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:48.018117 9512 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:47:48.018216 9512 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:47:48.028280 9512 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:37315
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:34141
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:34141
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:48.029688 9512 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:48.031059 9512 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:48.044554 9521 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:48.044598 9518 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:48.044766 9517 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:48.045590 9512 server_base.cc:1034] running on GCE node
I20260810 14:47:48.046392 9512 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:48.047902 9512 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:48.049109 9512 hybrid_clock.cc:648] HybridClock initialized: now 1786373268049069 us; error 30 us; skew 500 ppm
I20260810 14:47:48.049472 9512 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:48 dist-test-slave-vkjp krb5kdc[9494](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373268, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:48.052625 9512 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:47:48.054116 9512 webserver.cc:460] Webserver started at http://127.24.215.190:36819/ using document root <none> and password file <none>
I20260810 14:47:48.054818 9512 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:48.054970 9512 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:48.055341 9512 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:47:48.058391 9512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "8e6396cb2b814b7da586bbdde2986e1f"
format_stamp: "Formatted at 2026-08-10 14:47:48 on dist-test-slave-vkjp"
server_key: "9d2ce647fd6ad938617493ea07373047"
server_key_iv: "171e5ef7c8a91ba1a7add641b09b1ef8"
server_key_version: "encryptionkey@0"
I20260810 14:47:48.059226 9512 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "8e6396cb2b814b7da586bbdde2986e1f"
format_stamp: "Formatted at 2026-08-10 14:47:48 on dist-test-slave-vkjp"
server_key: "9d2ce647fd6ad938617493ea07373047"
server_key_iv: "171e5ef7c8a91ba1a7add641b09b1ef8"
server_key_version: "encryptionkey@0"
I20260810 14:47:48.064512 9512 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260810 14:47:48.067946 9529 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:48.069334 9512 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:47:48.069545 9512 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "8e6396cb2b814b7da586bbdde2986e1f"
format_stamp: "Formatted at 2026-08-10 14:47:48 on dist-test-slave-vkjp"
server_key: "9d2ce647fd6ad938617493ea07373047"
server_key_iv: "171e5ef7c8a91ba1a7add641b09b1ef8"
server_key_version: "encryptionkey@0"
I20260810 14:47:48.069764 9512 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:48.087680 9512 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:48.091697 9512 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:48.091987 9512 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:48.111047 9512 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:34141
I20260810 14:47:48.111129 9584 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:34141 every 8 connection(s)
I20260810 14:47:48.112493 9512 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:47:48.113216 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 9512
I20260810 14:47:48.113480 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:47:48.114231 25438 external_mini_cluster.cc:1428] Setting key b706cc6dd740f3124b5eb9c02d1d1a6d
I20260810 14:47:48.117091 9585 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 14:47:48 dist-test-slave-vkjp krb5kdc[9494](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373267, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:47:48.129766 9585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f: Bootstrap starting.
I20260810 14:47:48.133867 9585 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f: Neither blocks nor log segments found. Creating new log.
I20260810 14:47:48.135239 9585 log.cc:824] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f: Log is configured to *not* fsync() on all Append() calls
I20260810 14:47:48.138702 9585 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f: No bootstrap required, opened a new log
I20260810 14:47:48.139853 9588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:48.116238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41214 (local address 127.24.215.190:34141)
0810 14:47:48.117144 (+ 906us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:48.117160 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:48.117210 (+ 50us) server_negotiation.cc:407] Connection header received
0810 14:47:48.119637 (+ 2427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:48.119665 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:48.120150 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:48.120634 (+ 484us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:48.122033 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:48.124138 (+ 2105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:48.125432 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:48.126451 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:48.126639 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:48.130881 (+ 4242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:48.130891 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:48.130908 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:48.130957 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:48.134987 (+ 4030us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:48.135866 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:48.135877 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:48.135889 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:48.136076 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:48.136575 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:48.136580 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:48.136583 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:48.137293 (+ 710us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:48.137599 (+ 306us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:48.138030 (+ 431us) server_negotiation.cc:299] Negotiation successful
0810 14:47:48.138492 (+ 462us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":655,"thread_start_us":402,"threads_started":1}
I20260810 14:47:48.143819 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:47:48.147567 9585 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6396cb2b814b7da586bbdde2986e1f" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34141 } }
I20260810 14:47:48.147964 9585 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:47:48.148094 9585 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e6396cb2b814b7da586bbdde2986e1f, State: Initialized, Role: FOLLOWER
I20260810 14:47:48.148694 9585 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6396cb2b814b7da586bbdde2986e1f" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34141 } }
I20260810 14:47:48.148932 9585 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:47:48.149088 9585 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:47:48.149267 9585 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:47:48.151512 9585 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6396cb2b814b7da586bbdde2986e1f" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34141 } }
I20260810 14:47:48.152027 9585 leader_election.cc:302] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8e6396cb2b814b7da586bbdde2986e1f; no voters:
I20260810 14:47:48.152621 9585 leader_election.cc:288] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:47:48.152779 9590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:47:48.153496 9590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [term 1 LEADER]: Becoming Leader. State: Replica: 8e6396cb2b814b7da586bbdde2986e1f, State: Running, Role: LEADER
I20260810 14:47:48.154001 9590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6396cb2b814b7da586bbdde2986e1f" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34141 } }
I20260810 14:47:48.154729 9585 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:47:48.156797 9591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8e6396cb2b814b7da586bbdde2986e1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6396cb2b814b7da586bbdde2986e1f" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34141 } } }
I20260810 14:47:48.156774 9592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8e6396cb2b814b7da586bbdde2986e1f. Latest consensus state: current_term: 1 leader_uuid: "8e6396cb2b814b7da586bbdde2986e1f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8e6396cb2b814b7da586bbdde2986e1f" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34141 } } }
I20260810 14:47:48.157238 9591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [sys.catalog]: This master's current role is: LEADER
I20260810 14:47:48.157400 9592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f [sys.catalog]: This master's current role is: LEADER
I20260810 14:47:48.158833 9595 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:47:48.164259 9595 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:47:48.173012 9595 catalog_manager.cc:1347] Generated new cluster ID: 7433c9ce4b4b445abd0bf9b060ab09ad
I20260810 14:47:48.173144 9595 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:47:48.184122 9595 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:47:48.185945 9595 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:47:48.200465 9595 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8e6396cb2b814b7da586bbdde2986e1f: Generated new TSK 0
I20260810 14:47:48.201527 9595 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:47:48.256323 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 9512
I20260810 14:47:48.257552 9580 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:48.355259 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 9512
2026-08-10T14:47:48Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2163 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:47:48 dist-test-slave-vkjp krb5kdc[9615](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:47:48 dist-test-slave-vkjp krb5kdc[9615](info): set up 2 sockets
Aug 10 14:47:48 dist-test-slave-vkjp krb5kdc[9615](info): commencing operation
krb5kdc: starting...
W20260810 14:47:49.561412 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.002s sys 0.029s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:47:49 dist-test-slave-vkjp krb5kdc[9615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:47:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:47:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:47:49.835366 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:35483
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:35483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:47:50.197870 9629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:50.198225 9629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:50.198401 9629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:50.205772 9629 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:47:50.205933 9629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:50.206053 9629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:50.206156 9629 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:47:50.206243 9629 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:47:50.216202 9629 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:35483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:35483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:50.217631 9629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:50.218983 9629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:50.231431 9634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:50.231490 9637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:50.231513 9635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:50.233107 9629 server_base.cc:1034] running on GCE node
I20260810 14:47:50.234005 9629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:50.235594 9629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:50.236894 9629 hybrid_clock.cc:648] HybridClock initialized: now 1786373270236798 us; error 50 us; skew 500 ppm
I20260810 14:47:50.237432 9629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:50 dist-test-slave-vkjp krb5kdc[9615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373270, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:50.240984 9629 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:47:50.242517 9629 webserver.cc:460] Webserver started at http://127.24.215.190:45995/ using document root <none> and password file <none>
I20260810 14:47:50.243299 9629 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:50.243453 9629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:50.243773 9629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:47:50.246768 9629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "8ba6f5e397d141498c43714f9744daf4"
format_stamp: "Formatted at 2026-08-10 14:47:50 on dist-test-slave-vkjp"
server_key: "ef8dee847a0a00e82eaed55f29b61799"
server_key_iv: "a676982ec185e760a5dbd51e6a80fc14"
server_key_version: "encryptionkey@0"
I20260810 14:47:50.247592 9629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "8ba6f5e397d141498c43714f9744daf4"
format_stamp: "Formatted at 2026-08-10 14:47:50 on dist-test-slave-vkjp"
server_key: "ef8dee847a0a00e82eaed55f29b61799"
server_key_iv: "a676982ec185e760a5dbd51e6a80fc14"
server_key_version: "encryptionkey@0"
I20260810 14:47:50.252734 9629 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:47:50.256275 9646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:50.257606 9629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:47:50.257790 9629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "8ba6f5e397d141498c43714f9744daf4"
format_stamp: "Formatted at 2026-08-10 14:47:50 on dist-test-slave-vkjp"
server_key: "ef8dee847a0a00e82eaed55f29b61799"
server_key_iv: "a676982ec185e760a5dbd51e6a80fc14"
server_key_version: "encryptionkey@0"
I20260810 14:47:50.257993 9629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:50.280202 9629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:50.284696 9629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:50.285015 9629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:50.304560 9629 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:35483
I20260810 14:47:50.304605 9704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:35483 every 8 connection(s)
I20260810 14:47:50.305994 9629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:47:50.309818 9705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:47:50.312328 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 9629
I20260810 14:47:50.312577 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:47:50.313407 25438 external_mini_cluster.cc:1428] Setting key c5a7c4ae50202ac20484ff75039c3db3
I20260810 14:47:50.322861 9705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4: Bootstrap starting.
I20260810 14:47:50.326769 9705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4: Neither blocks nor log segments found. Creating new log.
I20260810 14:47:50.328176 9705 log.cc:824] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4: Log is configured to *not* fsync() on all Append() calls
Aug 10 14:47:50 dist-test-slave-vkjp krb5kdc[9615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:47:50.331497 9705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4: No bootstrap required, opened a new log
I20260810 14:47:50.337620 9708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:50.315646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34836 (local address 127.24.215.190:35483)
0810 14:47:50.316485 (+ 839us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:50.316498 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:50.316539 (+ 41us) server_negotiation.cc:407] Connection header received
0810 14:47:50.318177 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:50.318199 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:50.318577 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:50.318905 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:50.320880 (+ 1975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:50.322181 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:50.324024 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:50.324714 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:50.324841 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:50.330093 (+ 5252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:50.330100 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:50.330115 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:50.330158 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:50.333391 (+ 3233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:50.334135 (+ 744us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:50.334141 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:50.334147 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:50.334316 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:50.334741 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:50.334745 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:50.334747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:50.335339 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:50.335705 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:50.336169 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 14:47:50.336606 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":633,"thread_start_us":409,"threads_started":1}
I20260810 14:47:50.339532 9705 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } }
I20260810 14:47:50.339920 9705 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:47:50.340044 9705 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ba6f5e397d141498c43714f9744daf4, State: Initialized, Role: FOLLOWER
I20260810 14:47:50.340670 9705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } }
I20260810 14:47:50.340963 9705 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:47:50.341125 9705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:47:50.341302 9705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:47:50.343991 9705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } }
I20260810 14:47:50.344511 9705 leader_election.cc:302] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ba6f5e397d141498c43714f9744daf4; no voters:
I20260810 14:47:50.345093 9705 leader_election.cc:288] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:47:50.345227 9710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:47:50.346006 9710 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 1 LEADER]: Becoming Leader. State: Replica: 8ba6f5e397d141498c43714f9744daf4, State: Running, Role: LEADER
I20260810 14:47:50.346509 9710 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } }
I20260810 14:47:50.347343 9705 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:47:50.349442 9711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8ba6f5e397d141498c43714f9744daf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } } }
I20260810 14:47:50.349866 9711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [sys.catalog]: This master's current role is: LEADER
I20260810 14:47:50.350222 9715 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ba6f5e397d141498c43714f9744daf4. Latest consensus state: current_term: 1 leader_uuid: "8ba6f5e397d141498c43714f9744daf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } } }
I20260810 14:47:50.350639 9715 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [sys.catalog]: This master's current role is: LEADER
I20260810 14:47:50.356289 9719 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:47:50.360829 9719 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:47:50.370777 9719 catalog_manager.cc:1347] Generated new cluster ID: 2a22cd9fac2f4587b8b4354859ac4d89
I20260810 14:47:50.370918 9719 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:47:50.380534 9719 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:47:50.381850 9719 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:47:50.391520 9719 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4: Generated new TSK 0
I20260810 14:47:50.392448 9719 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:47:50.466807 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:35483
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:47:50.811388 9737 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:50.811729 9737 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:50.811915 9737 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:50.819146 9737 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:50.819305 9737 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:50.819479 9737 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:47:50.829208 9737 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:35483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:50.830520 9737 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:50.831908 9737 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:50.846045 9744 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:50.846040 9747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:50.846058 9745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:50.847246 9737 server_base.cc:1034] running on GCE node
I20260810 14:47:50.847936 9737 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:50.848970 9737 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:50.850204 9737 hybrid_clock.cc:648] HybridClock initialized: now 1786373270850141 us; error 64 us; skew 500 ppm
I20260810 14:47:50.850529 9737 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:50 dist-test-slave-vkjp krb5kdc[9615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373270, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:50.853696 9737 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:47:50.855314 9737 webserver.cc:460] Webserver started at http://127.24.215.129:35093/ using document root <none> and password file <none>
I20260810 14:47:50.856065 9737 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:50.856220 9737 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:50.856544 9737 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:47:50.859516 9737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "1cfe7b76354a4695b8a966262e90ba63"
format_stamp: "Formatted at 2026-08-10 14:47:50 on dist-test-slave-vkjp"
server_key: "0cb3cbbf06983095784cacfef4d2917c"
server_key_iv: "5590057fd66155a92c534201aff75d8f"
server_key_version: "encryptionkey@0"
I20260810 14:47:50.860260 9737 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "1cfe7b76354a4695b8a966262e90ba63"
format_stamp: "Formatted at 2026-08-10 14:47:50 on dist-test-slave-vkjp"
server_key: "0cb3cbbf06983095784cacfef4d2917c"
server_key_iv: "5590057fd66155a92c534201aff75d8f"
server_key_version: "encryptionkey@0"
I20260810 14:47:50.865852 9737 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260810 14:47:50.869393 9754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:50.870774 9737 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:47:50.870955 9737 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "1cfe7b76354a4695b8a966262e90ba63"
format_stamp: "Formatted at 2026-08-10 14:47:50 on dist-test-slave-vkjp"
server_key: "0cb3cbbf06983095784cacfef4d2917c"
server_key_iv: "5590057fd66155a92c534201aff75d8f"
server_key_version: "encryptionkey@0"
I20260810 14:47:50.871223 9737 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:50.883044 9737 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:50.887457 9737 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:50.887807 9737 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:50.888835 9737 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:47:50.890296 9737 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:47:50.890412 9737 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:50.890571 9737 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:47:50.890646 9737 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:50.925077 9737 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:32857
I20260810 14:47:50.925124 9881 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:32857 every 8 connection(s)
I20260810 14:47:50.926401 9737 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:47:50.935957 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 9737
I20260810 14:47:50.936175 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:47:50.937059 25438 external_mini_cluster.cc:1428] Setting key 2699e1952cb21abf526686d4def8bb56
Aug 10 14:47:50 dist-test-slave-vkjp krb5kdc[9615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373270, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:47:50.948505 9885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:50.929078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:47065 (local address 127.24.215.190:35483)
0810 14:47:50.929599 (+ 521us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:50.929606 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:50.930499 (+ 893us) server_negotiation.cc:407] Connection header received
0810 14:47:50.932313 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:50.932322 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:50.932421 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:50.932560 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:50.933953 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:50.934705 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:50.936270 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:50.936945 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:50.937100 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:50.941196 (+ 4096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:50.941206 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:50.941213 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:50.941260 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:50.944914 (+ 3654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:50.945666 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:50.945674 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:50.945680 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:50.945837 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:50.946262 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:50.946270 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:50.946277 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:50.946677 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:50.946833 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:50.947619 (+ 786us) server_negotiation.cc:299] Negotiation successful
0810 14:47:50.947925 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":241,"threads_started":1}
I20260810 14:47:50.949195 9886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:50.929611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:47065)
0810 14:47:50.930334 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:50.930371 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:50.931992 (+ 1621us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:50.932749 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:50.932756 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:50.933248 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:50.933795 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:50.933812 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:50.934844 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:50.934847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:50.936104 (+ 1257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:50.936133 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:50.937194 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:50.937203 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:50.937426 (+ 223us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:50.938140 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:50.938188 (+ 48us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:50.940943 (+ 2755us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:50.945075 (+ 4132us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:50.945082 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:50.945095 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:50.945531 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:50.945954 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:50.945957 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:50.945959 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:50.946135 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:50.946799 (+ 664us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:50.946805 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:50.947195 (+ 390us) client_negotiation.cc:769] Sending connection context
0810 14:47:50.947446 (+ 251us) client_negotiation.cc:240] Negotiation successful
0810 14:47:50.947980 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":272,"threads_started":1}
I20260810 14:47:50.951000 9882 heartbeater.cc:347] Connected to a master server at 127.24.215.190:35483
I20260810 14:47:50.951383 9882 heartbeater.cc:464] Registering TS with master...
I20260810 14:47:50.952164 9882 heartbeater.cc:511] Master 127.24.215.190:35483 requested a full tablet report, sending...
I20260810 14:47:50.954596 9668 ts_manager.cc:194] Registered new tserver with Master: 1cfe7b76354a4695b8a966262e90ba63 (127.24.215.129:32857)
I20260810 14:47:50.956878 9668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:47065
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:47:51.002844 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:35483
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:47:51.347409 9889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:51.347785 9889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:51.347975 9889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:51.355430 9889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:51.355536 9889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:51.355708 9889 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:47:51.365811 9889 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:35483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:51.367177 9889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:51.368556 9889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:51.382095 9897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:51.382154 9895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:51.382167 9899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:51.382884 9889 server_base.cc:1034] running on GCE node
I20260810 14:47:51.383554 9889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:51.384549 9889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:51.385757 9889 hybrid_clock.cc:648] HybridClock initialized: now 1786373271385694 us; error 56 us; skew 500 ppm
I20260810 14:47:51.386102 9889 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:51 dist-test-slave-vkjp krb5kdc[9615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373271, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:51.389442 9889 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:47:51.390899 9889 webserver.cc:460] Webserver started at http://127.24.215.130:46843/ using document root <none> and password file <none>
I20260810 14:47:51.391736 9889 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:51.391889 9889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:51.392206 9889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:47:51.395160 9889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "7770247d0e9043c1aff1592d4d084f4d"
format_stamp: "Formatted at 2026-08-10 14:47:51 on dist-test-slave-vkjp"
server_key: "e2fdbda65d52d001cdb3dfbf41fdf6bf"
server_key_iv: "c92460cccee1a4086388838761430836"
server_key_version: "encryptionkey@0"
I20260810 14:47:51.395936 9889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "7770247d0e9043c1aff1592d4d084f4d"
format_stamp: "Formatted at 2026-08-10 14:47:51 on dist-test-slave-vkjp"
server_key: "e2fdbda65d52d001cdb3dfbf41fdf6bf"
server_key_iv: "c92460cccee1a4086388838761430836"
server_key_version: "encryptionkey@0"
I20260810 14:47:51.401037 9889 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:47:51.404481 9907 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:51.405777 9889 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:47:51.405973 9889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "7770247d0e9043c1aff1592d4d084f4d"
format_stamp: "Formatted at 2026-08-10 14:47:51 on dist-test-slave-vkjp"
server_key: "e2fdbda65d52d001cdb3dfbf41fdf6bf"
server_key_iv: "c92460cccee1a4086388838761430836"
server_key_version: "encryptionkey@0"
I20260810 14:47:51.406167 9889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:51.468964 9889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:51.472991 9889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:51.473291 9889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:51.474320 9889 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:47:51.475800 9889 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:47:51.475875 9889 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:51.476039 9889 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:47:51.476096 9889 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:51.510344 9889 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:45417
I20260810 14:47:51.510365 10037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:45417 every 8 connection(s)
I20260810 14:47:51.511873 9889 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:47:51.520991 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 9889
I20260810 14:47:51.521173 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:47:51.521651 25438 external_mini_cluster.cc:1428] Setting key c8d7978c7778fa2be799f5956bd7dc95
Aug 10 14:47:51 dist-test-slave-vkjp krb5kdc[9615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373271, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:47:51.533440 10040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:51.514609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:35369 (local address 127.24.215.190:35483)
0810 14:47:51.515055 (+ 446us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:51.515060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:51.516117 (+ 1057us) server_negotiation.cc:407] Connection header received
0810 14:47:51.517973 (+ 1856us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:51.517979 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:51.518075 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:51.518204 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:51.519832 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:51.520604 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:51.522604 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:51.523065 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:51.523255 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:51.527458 (+ 4203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:51.527464 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:51.527468 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:51.527507 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:51.530262 (+ 2755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:51.531018 (+ 756us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:51.531023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:51.531025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:51.531216 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:51.531679 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:51.531684 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:51.531688 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:51.531956 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:51.532113 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:51.532828 (+ 715us) server_negotiation.cc:299] Negotiation successful
0810 14:47:51.533069 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":218,"threads_started":1}
I20260810 14:47:51.534713 10042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:51.515239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:35369)
0810 14:47:51.515949 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:51.515995 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:51.517683 (+ 1688us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:51.518457 (+ 774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:51.518466 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:51.518993 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:51.519647 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:51.519666 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:51.520804 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:51.520809 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:51.522367 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:51.522384 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:51.523354 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:51.523361 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:51.523617 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:51.524457 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:51.524483 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:51.527189 (+ 2706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:51.530421 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:51.530431 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:51.530448 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:51.530877 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:51.531331 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:51.531336 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:51.531340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:51.531548 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:51.532087 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:51.532095 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:51.532458 (+ 363us) client_negotiation.cc:769] Sending connection context
0810 14:47:51.532723 (+ 265us) client_negotiation.cc:240] Negotiation successful
0810 14:47:51.533283 (+ 560us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":468,"thread_start_us":266,"threads_started":1}
I20260810 14:47:51.536320 10038 heartbeater.cc:347] Connected to a master server at 127.24.215.190:35483
I20260810 14:47:51.536661 10038 heartbeater.cc:464] Registering TS with master...
I20260810 14:47:51.537671 10038 heartbeater.cc:511] Master 127.24.215.190:35483 requested a full tablet report, sending...
I20260810 14:47:51.539498 9668 ts_manager.cc:194] Registered new tserver with Master: 7770247d0e9043c1aff1592d4d084f4d (127.24.215.130:45417)
I20260810 14:47:51.540800 9668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:35369
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:47:51.587025 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:35483
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:47:51.925637 10045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:51.925925 10045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:51.926091 10045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:51.933499 10045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:51.933609 10045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:51.933831 10045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:47:51.943745 10045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:35483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:51.945068 10045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:51.946373 10045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260810 14:47:51.960196 9882 heartbeater.cc:503] Master 127.24.215.190:35483 was elected leader, sending a full tablet report...
W20260810 14:47:51.960184 10052 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:51.960158 10054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:51.960755 10051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:51.961234 10045 server_base.cc:1034] running on GCE node
I20260810 14:47:51.962023 10045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:51.963243 10045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:51.964480 10045 hybrid_clock.cc:648] HybridClock initialized: now 1786373271964403 us; error 64 us; skew 500 ppm
I20260810 14:47:51.964823 10045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:51 dist-test-slave-vkjp krb5kdc[9615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373271, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:51.968422 10045 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:47:51.969890 10045 webserver.cc:460] Webserver started at http://127.24.215.131:38219/ using document root <none> and password file <none>
I20260810 14:47:51.970551 10045 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:51.970654 10045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:51.970912 10045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:47:51.973994 10045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "5f800c59113d401dbece99261430a69e"
format_stamp: "Formatted at 2026-08-10 14:47:51 on dist-test-slave-vkjp"
server_key: "13146e610d1be4baf699389e3ee5443e"
server_key_iv: "b16cea9b57178707ab98b0b23f777a55"
server_key_version: "encryptionkey@0"
I20260810 14:47:51.974895 10045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "5f800c59113d401dbece99261430a69e"
format_stamp: "Formatted at 2026-08-10 14:47:51 on dist-test-slave-vkjp"
server_key: "13146e610d1be4baf699389e3ee5443e"
server_key_iv: "b16cea9b57178707ab98b0b23f777a55"
server_key_version: "encryptionkey@0"
I20260810 14:47:51.980269 10045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260810 14:47:51.983899 10064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:51.985235 10045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:47:51.985378 10045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "5f800c59113d401dbece99261430a69e"
format_stamp: "Formatted at 2026-08-10 14:47:51 on dist-test-slave-vkjp"
server_key: "13146e610d1be4baf699389e3ee5443e"
server_key_iv: "b16cea9b57178707ab98b0b23f777a55"
server_key_version: "encryptionkey@0"
I20260810 14:47:51.985584 10045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:52.004848 10045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:52.009306 10045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:52.009572 10045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:52.010643 10045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:47:52.012161 10045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:47:52.012239 10045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:52.012331 10045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:47:52.012374 10045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:52.046448 10045 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:34537
I20260810 14:47:52.046490 10192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:34537 every 8 connection(s)
I20260810 14:47:52.047919 10045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:47:52.055681 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 10045
I20260810 14:47:52.055897 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:47:52.056565 25438 external_mini_cluster.cc:1428] Setting key 393e444b2731ce90dcb312b414cf6e14
Aug 10 14:47:52 dist-test-slave-vkjp krb5kdc[9615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373271, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:47:52.068715 10196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.050522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:60569 (local address 127.24.215.190:35483)
0810 14:47:52.050945 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:52.050950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:52.052033 (+ 1083us) server_negotiation.cc:407] Connection header received
0810 14:47:52.053716 (+ 1683us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:52.053721 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:52.053789 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:52.053891 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:52.055391 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.056647 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.059133 (+ 2486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.059552 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.059699 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.063352 (+ 3653us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:52.063358 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:52.063361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:52.063402 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:52.065968 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:52.066671 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:52.066675 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:52.066678 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:52.066797 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:52.067234 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:52.067237 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:52.067239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:52.067499 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:52.067624 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:52.068253 (+ 629us) server_negotiation.cc:299] Negotiation successful
0810 14:47:52.068443 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":201,"threads_started":1}
I20260810 14:47:52.069619 10197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.051128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:60569)
0810 14:47:52.051864 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:52.051911 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:52.053468 (+ 1557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:52.054100 (+ 632us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:52.054107 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:52.054601 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:52.055201 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:52.055221 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.056811 (+ 1590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.056816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:52.058952 (+ 2136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:52.058969 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.059802 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.059809 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:52.060056 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.060667 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:52.060695 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:52.063141 (+ 2446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:52.066108 (+ 2967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:52.066115 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:52.066127 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:52.066562 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:52.066888 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:52.066891 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:52.066893 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:52.067073 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:52.067611 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:52.067616 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:52.067937 (+ 321us) client_negotiation.cc:769] Sending connection context
0810 14:47:52.068163 (+ 226us) client_negotiation.cc:240] Negotiation successful
0810 14:47:52.068541 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":272,"threads_started":1}
I20260810 14:47:52.070999 10193 heartbeater.cc:347] Connected to a master server at 127.24.215.190:35483
I20260810 14:47:52.071389 10193 heartbeater.cc:464] Registering TS with master...
I20260810 14:47:52.072345 10193 heartbeater.cc:511] Master 127.24.215.190:35483 requested a full tablet report, sending...
I20260810 14:47:52.073858 9668 ts_manager.cc:194] Registered new tserver with Master: 5f800c59113d401dbece99261430a69e (127.24.215.131:34537)
I20260810 14:47:52.074702 9668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:60569
I20260810 14:47:52.085146 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:47:52.102850 10196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.090124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39020 (local address 127.24.215.190:35483)
0810 14:47:52.090316 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:52.090324 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:52.090792 (+ 468us) server_negotiation.cc:407] Connection header received
0810 14:47:52.091020 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:52.091025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:52.091164 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:52.091319 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:52.092739 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.093482 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.094761 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.095234 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.095337 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.097630 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:52.097635 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:52.097638 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:52.097670 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:52.100362 (+ 2692us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:52.101086 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:52.101093 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:52.101098 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:52.101233 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:52.101667 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:52.101670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:52.101672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:52.101974 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:52.102147 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:52.102454 (+ 307us) server_negotiation.cc:299] Negotiation successful
0810 14:47:52.102624 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:47:52.106963 9668 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:47:52.114312 9668 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:47:52.141217 10211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.128198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:34537 (local address 127.0.0.1:50348)
0810 14:47:52.129227 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:52.129278 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:52.129492 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:52.133450 (+ 3958us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:52.133456 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:52.133491 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:47:52.134015 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:52.134033 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.135828 (+ 1795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.135834 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:52.138278 (+ 2444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:52.138291 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.140074 (+ 1783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.140081 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:52.140294 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.140353 (+ 59us) client_negotiation.cc:769] Sending connection context
0810 14:47:52.140489 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 14:47:52.140681 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":107,"thread_start_us":1628,"threads_started":1}
I20260810 14:47:52.141661 10217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.130144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56432 (local address 127.24.215.130:45417)
0810 14:47:52.131563 (+ 1419us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:52.131576 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:52.132315 (+ 739us) server_negotiation.cc:407] Connection header received
0810 14:47:52.132441 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:52.132449 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:52.132697 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:52.133587 (+ 890us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:47:52.134308 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.135842 (+ 1534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.138797 (+ 2955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.140265 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.140466 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.140627 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:52.140890 (+ 263us) server_negotiation.cc:299] Negotiation successful
0810 14:47:52.141087 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1242,"thread_start_us":1134,"threads_started":1}
I20260810 14:47:52.141987 10212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.128300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:32857 (local address 127.0.0.1:54130)
0810 14:47:52.130171 (+ 1871us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:52.130194 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:52.130337 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:52.132936 (+ 2599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:52.132944 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:52.132974 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:47:52.133505 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:52.133521 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.135702 (+ 2181us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.135707 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:52.138632 (+ 2925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:52.138647 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.140859 (+ 2212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.140866 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:52.141056 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.141095 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 14:47:52.141193 (+ 98us) client_negotiation.cc:240] Negotiation successful
0810 14:47:52.141479 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1710,"mutex_wait_us":156,"thread_start_us":430,"threads_started":1}
I20260810 14:47:52.142015 10213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.128426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50348 (local address 127.24.215.131:34537)
0810 14:47:52.132861 (+ 4435us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:52.132878 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:52.132910 (+ 32us) server_negotiation.cc:407] Connection header received
0810 14:47:52.132998 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:52.133004 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:52.133226 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:52.134237 (+ 1011us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:47:52.134498 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.135644 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.138447 (+ 2803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.139868 (+ 1421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.140081 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.140263 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:52.141466 (+ 1203us) server_negotiation.cc:299] Negotiation successful
0810 14:47:52.141644 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4267,"thread_start_us":4118,"threads_started":1}
I20260810 14:47:52.142586 10216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.129628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:45417 (local address 127.0.0.1:56432)
0810 14:47:52.132028 (+ 2400us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:52.132047 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:52.132181 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:52.133186 (+ 1005us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:52.133193 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:52.133228 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:47:52.134112 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:52.134128 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.136027 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.136034 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:52.138635 (+ 2601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:52.138647 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.140501 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.140507 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:52.140679 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.140714 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 14:47:52.140823 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 14:47:52.142291 (+ 1468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2273,"mutex_wait_us":302,"thread_start_us":2129,"threads_started":1}
I20260810 14:47:52.143422 10215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.128784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54130 (local address 127.24.215.129:32857)
0810 14:47:52.132395 (+ 3611us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:52.132402 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:52.132427 (+ 25us) server_negotiation.cc:407] Connection header received
0810 14:47:52.132516 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:52.132522 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:52.132697 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:52.132907 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:47:52.133790 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.135503 (+ 1713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.139219 (+ 3716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.140302 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.140531 (+ 229us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.140686 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:52.142862 (+ 2176us) server_negotiation.cc:299] Negotiation successful
0810 14:47:52.143044 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3428,"thread_start_us":3296,"threads_started":1}
I20260810 14:47:52.145756 10117 tablet_service.cc:1467] Processing CreateTablet for tablet 465ef1d24156437d9cdb955ea7abe967 (DEFAULT_TABLE table=test-table [id=bd2daff13b604ef287401fb3821504d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:47:52.146126 9960 tablet_service.cc:1467] Processing CreateTablet for tablet 465ef1d24156437d9cdb955ea7abe967 (DEFAULT_TABLE table=test-table [id=bd2daff13b604ef287401fb3821504d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:47:52.147373 10117 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 465ef1d24156437d9cdb955ea7abe967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:47:52.146894 9810 tablet_service.cc:1467] Processing CreateTablet for tablet 465ef1d24156437d9cdb955ea7abe967 (DEFAULT_TABLE table=test-table [id=bd2daff13b604ef287401fb3821504d8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:47:52.147558 9960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 465ef1d24156437d9cdb955ea7abe967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:47:52.148227 9810 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 465ef1d24156437d9cdb955ea7abe967. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:47:52.159663 10218 tablet_bootstrap.cc:492] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e: Bootstrap starting.
I20260810 14:47:52.162739 10219 tablet_bootstrap.cc:492] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63: Bootstrap starting.
I20260810 14:47:52.163658 10218 tablet_bootstrap.cc:654] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e: Neither blocks nor log segments found. Creating new log.
I20260810 14:47:52.165072 10218 log.cc:824] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:47:52.165966 10221 tablet_bootstrap.cc:492] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d: Bootstrap starting.
I20260810 14:47:52.166639 10219 tablet_bootstrap.cc:654] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63: Neither blocks nor log segments found. Creating new log.
I20260810 14:47:52.167886 10219 log.cc:824] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63: Log is configured to *not* fsync() on all Append() calls
I20260810 14:47:52.169623 10221 tablet_bootstrap.cc:654] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d: Neither blocks nor log segments found. Creating new log.
I20260810 14:47:52.170092 10218 tablet_bootstrap.cc:492] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e: No bootstrap required, opened a new log
I20260810 14:47:52.170370 10218 ts_tablet_manager.cc:1397] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.003s
I20260810 14:47:52.170919 10221 log.cc:824] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:47:52.171005 10219 tablet_bootstrap.cc:492] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63: No bootstrap required, opened a new log
I20260810 14:47:52.171315 10219 ts_tablet_manager.cc:1397] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 14:47:52.173978 10221 tablet_bootstrap.cc:492] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d: No bootstrap required, opened a new log
I20260810 14:47:52.174230 10221 ts_tablet_manager.cc:1397] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 14:47:52.178467 10218 raft_consensus.cc:348] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } } peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } }
I20260810 14:47:52.178889 10218 raft_consensus.cc:374] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:47:52.178994 10218 raft_consensus.cc:729] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f800c59113d401dbece99261430a69e, State: Initialized, Role: FOLLOWER
I20260810 14:47:52.179792 10218 consensus_queue.cc:260] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } } peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } }
I20260810 14:47:52.181519 10193 heartbeater.cc:503] Master 127.24.215.190:35483 was elected leader, sending a full tablet report...
I20260810 14:47:52.181785 10219 raft_consensus.cc:348] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } } peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } }
I20260810 14:47:52.182184 10218 ts_tablet_manager.cc:1428] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260810 14:47:52.182193 10219 raft_consensus.cc:374] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:47:52.182279 10219 raft_consensus.cc:729] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cfe7b76354a4695b8a966262e90ba63, State: Initialized, Role: FOLLOWER
I20260810 14:47:52.182847 10219 consensus_queue.cc:260] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } } peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } }
I20260810 14:47:52.184592 10219 ts_tablet_manager.cc:1428] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63: Time spent starting tablet: real 0.013s user 0.012s sys 0.000s
I20260810 14:47:52.184871 10221 raft_consensus.cc:348] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } } peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } }
I20260810 14:47:52.185324 10221 raft_consensus.cc:374] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:47:52.185447 10221 raft_consensus.cc:729] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7770247d0e9043c1aff1592d4d084f4d, State: Initialized, Role: FOLLOWER
I20260810 14:47:52.185995 10221 consensus_queue.cc:260] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } } peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } }
I20260810 14:47:52.187757 10038 heartbeater.cc:503] Master 127.24.215.190:35483 was elected leader, sending a full tablet report...
I20260810 14:47:52.187901 10221 ts_tablet_manager.cc:1428] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d: Time spent starting tablet: real 0.014s user 0.015s sys 0.000s
W20260810 14:47:52.263955 10039 tablet.cc:2369] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:47:52.299611 10194 tablet.cc:2369] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:47:52.429056 9884 tablet.cc:2369] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:47:52.484220 10228 raft_consensus.cc:482] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:47:52.484527 10228 raft_consensus.cc:504] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } } peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } }
I20260810 14:47:52.486302 10228 leader_election.cc:288] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f800c59113d401dbece99261430a69e (127.24.215.131:34537), 1cfe7b76354a4695b8a966262e90ba63 (127.24.215.129:32857)
I20260810 14:47:52.493775 10215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.486862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:52499 (local address 127.24.215.129:32857)
0810 14:47:52.487075 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:52.487103 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:52.487904 (+ 801us) server_negotiation.cc:407] Connection header received
0810 14:47:52.488151 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:52.488158 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:52.488227 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:52.488330 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:47:52.488976 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.489809 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.491881 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.492703 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.492893 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.492986 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:52.493356 (+ 370us) server_negotiation.cc:299] Negotiation successful
0810 14:47:52.493504 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260810 14:47:52.493774 10213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.486816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:49063 (local address 127.24.215.131:34537)
0810 14:47:52.487056 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:52.487064 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:52.488056 (+ 992us) server_negotiation.cc:407] Connection header received
0810 14:47:52.488290 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:52.488301 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:52.488361 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:52.488465 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:47:52.489258 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.490086 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.491933 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.492746 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.492921 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.493006 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:52.493351 (+ 345us) server_negotiation.cc:299] Negotiation successful
0810 14:47:52.493494 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260810 14:47:52.494012 10230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.486876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:34537 (local address 127.24.215.130:49063)
0810 14:47:52.487939 (+ 1063us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:52.487954 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:52.488134 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:52.488483 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:52.488487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:52.488528 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:47:52.489102 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:52.489114 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.490229 (+ 1115us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.490235 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:52.491767 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:52.491779 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.492993 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.493000 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:52.493183 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.493216 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:47:52.493322 (+ 106us) client_negotiation.cc:240] Negotiation successful
0810 14:47:52.493489 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":942,"mutex_wait_us":32,"thread_start_us":686,"threads_started":1}
I20260810 14:47:52.494377 10229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.486664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:32857 (local address 127.24.215.130:52499)
0810 14:47:52.487781 (+ 1117us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:52.487803 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:52.487994 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:52.488348 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:52.488352 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:52.488375 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:47:52.488822 (+ 447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:52.488832 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.489943 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.489950 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:52.491699 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:52.491715 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.492975 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.492982 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:52.493180 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.493211 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 14:47:52.493324 (+ 113us) client_negotiation.cc:240] Negotiation successful
0810 14:47:52.493498 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"mutex_wait_us":82,"thread_start_us":605,"threads_started":1}
I20260810 14:47:52.495309 10140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "465ef1d24156437d9cdb955ea7abe967" candidate_uuid: "7770247d0e9043c1aff1592d4d084f4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f800c59113d401dbece99261430a69e" is_pre_election: true
I20260810 14:47:52.495534 9831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "465ef1d24156437d9cdb955ea7abe967" candidate_uuid: "7770247d0e9043c1aff1592d4d084f4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cfe7b76354a4695b8a966262e90ba63" is_pre_election: true
I20260810 14:47:52.495975 10140 raft_consensus.cc:2454] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7770247d0e9043c1aff1592d4d084f4d in term 0.
I20260810 14:47:52.496081 9831 raft_consensus.cc:2454] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7770247d0e9043c1aff1592d4d084f4d in term 0.
I20260810 14:47:52.496709 9909 leader_election.cc:302] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f800c59113d401dbece99261430a69e, 7770247d0e9043c1aff1592d4d084f4d; no voters:
I20260810 14:47:52.497179 10228 raft_consensus.cc:2781] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:47:52.497354 10228 raft_consensus.cc:482] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:47:52.497485 10228 raft_consensus.cc:3037] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:47:52.499876 10228 raft_consensus.cc:504] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } } peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } }
I20260810 14:47:52.500515 10228 leader_election.cc:288] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [CANDIDATE]: Term 1 election: Requested vote from peers 5f800c59113d401dbece99261430a69e (127.24.215.131:34537), 1cfe7b76354a4695b8a966262e90ba63 (127.24.215.129:32857)
I20260810 14:47:52.501066 10140 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "465ef1d24156437d9cdb955ea7abe967" candidate_uuid: "7770247d0e9043c1aff1592d4d084f4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f800c59113d401dbece99261430a69e"
I20260810 14:47:52.501062 9831 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "465ef1d24156437d9cdb955ea7abe967" candidate_uuid: "7770247d0e9043c1aff1592d4d084f4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cfe7b76354a4695b8a966262e90ba63"
I20260810 14:47:52.501274 9831 raft_consensus.cc:3037] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:47:52.501279 10140 raft_consensus.cc:3037] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:47:52.503607 9831 raft_consensus.cc:2454] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7770247d0e9043c1aff1592d4d084f4d in term 1.
I20260810 14:47:52.503633 10140 raft_consensus.cc:2454] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7770247d0e9043c1aff1592d4d084f4d in term 1.
I20260810 14:47:52.504119 9910 leader_election.cc:302] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cfe7b76354a4695b8a966262e90ba63, 7770247d0e9043c1aff1592d4d084f4d; no voters:
I20260810 14:47:52.504456 10228 raft_consensus.cc:2781] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:47:52.504894 10228 raft_consensus.cc:686] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [term 1 LEADER]: Becoming Leader. State: Replica: 7770247d0e9043c1aff1592d4d084f4d, State: Running, Role: LEADER
I20260810 14:47:52.505440 10228 consensus_queue.cc:237] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } } peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } }
I20260810 14:47:52.510030 9668 catalog_manager.cc:5495] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7770247d0e9043c1aff1592d4d084f4d (127.24.215.130). New cstate: current_term: 1 leader_uuid: "7770247d0e9043c1aff1592d4d084f4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } health_report { overall_health: HEALTHY } } }
I20260810 14:47:52.599260 10217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.593609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56446 (local address 127.24.215.130:45417)
0810 14:47:52.593796 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:52.593801 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:52.593828 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:47:52.593919 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:52.593923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:52.593999 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:52.594130 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:47:52.594536 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.595362 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.596807 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.597274 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.597381 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.598072 (+ 691us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:47:52.598603 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:47:52.598731 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:52.598866 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 14:47:52.598950 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:47:52.608062 10140 raft_consensus.cc:1261] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e [term 1 FOLLOWER]: Refusing update from remote peer 7770247d0e9043c1aff1592d4d084f4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:47:52.608060 9831 raft_consensus.cc:1261] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63 [term 1 FOLLOWER]: Refusing update from remote peer 7770247d0e9043c1aff1592d4d084f4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:47:52.609143 10231 consensus_queue.cc:1035] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:47:52.609525 10228 consensus_queue.cc:1035] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:47:52.622587 10234 mvcc.cc:205] Tried to move back new op lower bound from 7316984924592635904 to 7316984924186345472. Current Snapshot: MvccSnapshot[applied={T|T < 7316984924592635904}]
I20260810 14:47:52.633965 9667 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:47:52.634315 9667 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:47:52.638398 9667 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4: Sending DeleteTablet for 3 replicas of tablet 465ef1d24156437d9cdb955ea7abe967
I20260810 14:47:52.639683 9810 tablet_service.cc:1514] Processing DeleteTablet for tablet 465ef1d24156437d9cdb955ea7abe967 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:47:52 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:54130
I20260810 14:47:52.639683 10117 tablet_service.cc:1514] Processing DeleteTablet for tablet 465ef1d24156437d9cdb955ea7abe967 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:47:52 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:50348
I20260810 14:47:52.640726 10244 tablet_replica.cc:330] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63: stopping tablet replica
I20260810 14:47:52.641130 9960 tablet_service.cc:1514] Processing DeleteTablet for tablet 465ef1d24156437d9cdb955ea7abe967 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:47:52 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:56432
I20260810 14:47:52.641258 10244 raft_consensus.cc:2229] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:47:52.641961 10244 raft_consensus.cc:2258] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:47:52.642329 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 9737
I20260810 14:47:52.644583 10244 ts_tablet_manager.cc:1905] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:47:52.647549 10245 tablet_replica.cc:330] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e: stopping tablet replica
I20260810 14:47:52.648165 10245 raft_consensus.cc:2229] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:47:52.648906 10246 tablet_replica.cc:330] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d: stopping tablet replica
I20260810 14:47:52.649273 10246 raft_consensus.cc:2229] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:47:52.649863 10246 raft_consensus.cc:2258] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:47:52.652482 10246 ts_tablet_manager.cc:1905] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:47:52.653833 10244 ts_tablet_manager.cc:1918] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:47:52.654026 10244 log.cc:1196] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/465ef1d24156437d9cdb955ea7abe967
I20260810 14:47:52.654623 10244 ts_tablet_manager.cc:1939] T 465ef1d24156437d9cdb955ea7abe967 P 1cfe7b76354a4695b8a966262e90ba63: Deleting consensus metadata
I20260810 14:47:52.655593 10245 raft_consensus.cc:2258] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:47:52.657989 10245 ts_tablet_manager.cc:1905] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:47:52.661656 10246 ts_tablet_manager.cc:1918] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:47:52.661914 10246 log.cc:1196] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/465ef1d24156437d9cdb955ea7abe967
Aug 10 14:47:52 dist-test-slave-vkjp krb5kdc[9615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:47:52.662786 10246 ts_tablet_manager.cc:1939] T 465ef1d24156437d9cdb955ea7abe967 P 7770247d0e9043c1aff1592d4d084f4d: Deleting consensus metadata
I20260810 14:47:52.664924 9649 catalog_manager.cc:4841] TS 7770247d0e9043c1aff1592d4d084f4d (127.24.215.130:45417): tablet 465ef1d24156437d9cdb955ea7abe967 (table test-table [id=bd2daff13b604ef287401fb3821504d8]) successfully deleted
I20260810 14:47:52.665863 10245 ts_tablet_manager.cc:1918] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:47:52.666026 10245 log.cc:1196] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/465ef1d24156437d9cdb955ea7abe967
I20260810 14:47:52.666554 10245 ts_tablet_manager.cc:1939] T 465ef1d24156437d9cdb955ea7abe967 P 5f800c59113d401dbece99261430a69e: Deleting consensus metadata
I20260810 14:47:52.667335 9649 catalog_manager.cc:4841] TS 1cfe7b76354a4695b8a966262e90ba63 (127.24.215.129:32857): tablet 465ef1d24156437d9cdb955ea7abe967 (table test-table [id=bd2daff13b604ef287401fb3821504d8]) successfully deleted
I20260810 14:47:52.668406 9648 catalog_manager.cc:4841] TS 5f800c59113d401dbece99261430a69e (127.24.215.131:34537): tablet 465ef1d24156437d9cdb955ea7abe967 (table test-table [id=bd2daff13b604ef287401fb3821504d8]) successfully deleted
I20260810 14:47:52.671353 10215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.644657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54146 (local address 127.24.215.129:32857)
0810 14:47:52.644884 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:52.644893 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:52.648671 (+ 3778us) server_negotiation.cc:407] Connection header received
0810 14:47:52.650505 (+ 1834us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:52.650511 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:52.650599 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:52.650673 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:52.652884 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.654068 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.656374 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.657107 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.657230 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.663794 (+ 6564us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:52.663811 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:52.663829 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:52.663874 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:52.667939 (+ 4065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:52.668924 (+ 985us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:52.668933 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:52.668936 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:52.669066 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:52.669490 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:52.669502 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:52.669505 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:52.669882 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:52.670210 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:52.670696 (+ 486us) server_negotiation.cc:299] Negotiation successful
0810 14:47:52.670950 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 14:47:52.672029 9876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:52.782063 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 9737
I20260810 14:47:52.804538 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 9889
Aug 10 14:47:52 dist-test-slave-vkjp krb5kdc[9615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:47:52.828733 10217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.805394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56458 (local address 127.24.215.130:45417)
0810 14:47:52.805592 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:52.805598 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:52.805621 (+ 23us) server_negotiation.cc:407] Connection header received
0810 14:47:52.805802 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:52.805808 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:52.805909 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:52.806025 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:52.807477 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.808747 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.810124 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.810870 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.811002 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.820588 (+ 9586us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:52.820611 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:52.820636 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:52.820693 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:52.825091 (+ 4398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:52.826019 (+ 928us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:52.826030 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:52.826034 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:52.826239 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:52.826713 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:52.826719 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:52.826723 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:52.827304 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:52.827684 (+ 380us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:52.827946 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 14:47:52.828286 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 14:47:52.829596 10031 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:52.944531 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 9889
I20260810 14:47:52.968360 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 10045
Aug 10 14:47:52 dist-test-slave-vkjp krb5kdc[9615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373269, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:47:52.983965 10213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:52.969182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50360 (local address 127.24.215.131:34537)
0810 14:47:52.969370 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:52.969374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:52.969423 (+ 49us) server_negotiation.cc:407] Connection header received
0810 14:47:52.969587 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:52.969592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:52.969659 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:52.969747 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:52.971029 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.971844 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.973165 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:52.973564 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:52.973663 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:52.978331 (+ 4668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:52.978346 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:52.978363 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:52.978409 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:52.981188 (+ 2779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:52.981878 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:52.981886 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:52.981888 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:52.982044 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:52.982473 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:52.982477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:52.982479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:52.982832 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:52.983069 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:52.983521 (+ 452us) server_negotiation.cc:299] Negotiation successful
0810 14:47:52.983688 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 14:47:52.984613 10186 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:53.075726 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 10045
I20260810 14:47:53.096367 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 9629
I20260810 14:47:53.097702 9700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:53.209241 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 9629
I20260810 14:47:53.234509 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:35483
--webserver_interface=127.24.215.190
--webserver_port=45995
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:35483
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:47:53.575155 10252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:53.575498 10252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:53.575670 10252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:53.583024 10252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:47:53.583230 10252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:53.583336 10252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:53.583420 10252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:47:53.583504 10252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:47:53.597805 10252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:35483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:35483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=45995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:53.599349 10252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:53.600720 10252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:53.625811 10259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:53.625773 10261 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:53.625768 10258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:53.626639 10252 server_base.cc:1034] running on GCE node
I20260810 14:47:53.627682 10252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:53.629483 10252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:53.630728 10252 hybrid_clock.cc:648] HybridClock initialized: now 1786373273630670 us; error 58 us; skew 500 ppm
I20260810 14:47:53.631204 10252 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:53 dist-test-slave-vkjp krb5kdc[9615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373273, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:53.635010 10252 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:47:53.636942 10252 webserver.cc:460] Webserver started at http://127.24.215.190:45995/ using document root <none> and password file <none>
I20260810 14:47:53.637813 10252 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:53.637986 10252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:53.644340 10252 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.002s
I20260810 14:47:53.647469 10269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:53.648808 10252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:47:53.649006 10252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "8ba6f5e397d141498c43714f9744daf4"
format_stamp: "Formatted at 2026-08-10 14:47:50 on dist-test-slave-vkjp"
server_key: "ef8dee847a0a00e82eaed55f29b61799"
server_key_iv: "a676982ec185e760a5dbd51e6a80fc14"
server_key_version: "encryptionkey@0"
I20260810 14:47:53.650116 10252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:53.668352 10252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:53.672646 10252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:53.672963 10252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:53.692833 10252 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:35483
I20260810 14:47:53.692850 10327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:35483 every 8 connection(s)
I20260810 14:47:53.694496 10252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:47:53.701107 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 10252
I20260810 14:47:53.702209 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:32857
--local_ip_for_outbound_sockets=127.24.215.129
--tserver_master_addrs=127.24.215.190:35483
--webserver_port=35093
--webserver_interface=127.24.215.129
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:47:53.702746 10328 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:47:53.708505 10328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4: Bootstrap starting.
I20260810 14:47:53.730757 10328 log.cc:824] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4: Log is configured to *not* fsync() on all Append() calls
I20260810 14:47:53.744277 10328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:47:53.744889 10328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4: Bootstrap complete.
I20260810 14:47:53.754707 10328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } }
I20260810 14:47:53.755724 10328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ba6f5e397d141498c43714f9744daf4, State: Initialized, Role: FOLLOWER
I20260810 14:47:53.756408 10328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } }
I20260810 14:47:53.756682 10328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:47:53.756816 10328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:47:53.757033 10328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:47:53.761133 10328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } }
I20260810 14:47:53.761754 10328 leader_election.cc:302] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ba6f5e397d141498c43714f9744daf4; no voters:
I20260810 14:47:53.762446 10328 leader_election.cc:288] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 14:47:53.762599 10332 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 14:47:53.763845 10332 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 2 LEADER]: Becoming Leader. State: Replica: 8ba6f5e397d141498c43714f9744daf4, State: Running, Role: LEADER
I20260810 14:47:53.764468 10332 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } }
I20260810 14:47:53.765019 10328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:47:53.767167 10334 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ba6f5e397d141498c43714f9744daf4. Latest consensus state: current_term: 2 leader_uuid: "8ba6f5e397d141498c43714f9744daf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } } }
I20260810 14:47:53.767561 10334 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [sys.catalog]: This master's current role is: LEADER
I20260810 14:47:53.767967 10333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "8ba6f5e397d141498c43714f9744daf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } } }
I20260810 14:47:53.770367 10333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [sys.catalog]: This master's current role is: LEADER
I20260810 14:47:53.778415 10341 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:47:53.787050 10341 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:47:53.788921 10341 catalog_manager.cc:1259] Loaded cluster ID: 2a22cd9fac2f4587b8b4354859ac4d89
I20260810 14:47:53.789057 10341 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:47:53.793851 10341 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260810 14:47:53.806303 10341 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260810 14:47:53.813730 10341 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:47:53.815196 10341 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4: Loaded TSK: 0
I20260810 14:47:53.816399 10341 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 14:47:54.088279 10330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:54.088578 10330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:54.088733 10330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:54.096098 10330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:54.096203 10330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:54.096385 10330 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:47:54.106778 10330 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:32857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=35093
--tserver_master_addrs=127.24.215.190:35483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:54.108197 10330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:54.109582 10330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:54.125566 10361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:54.125536 10362 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:54.126401 10364 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:54.126791 10330 server_base.cc:1034] running on GCE node
I20260810 14:47:54.127584 10330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:54.128705 10330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:54.129964 10330 hybrid_clock.cc:648] HybridClock initialized: now 1786373274129896 us; error 57 us; skew 500 ppm
I20260810 14:47:54.130317 10330 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:54 dist-test-slave-vkjp krb5kdc[9615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373274, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:54.133807 10330 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:47:54.135601 10330 webserver.cc:460] Webserver started at http://127.24.215.129:35093/ using document root <none> and password file <none>
I20260810 14:47:54.136330 10330 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:54.136498 10330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:54.142520 10330 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 14:47:54.146041 10374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:54.147781 10330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 14:47:54.147986 10330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "1cfe7b76354a4695b8a966262e90ba63"
format_stamp: "Formatted at 2026-08-10 14:47:50 on dist-test-slave-vkjp"
server_key: "0cb3cbbf06983095784cacfef4d2917c"
server_key_iv: "5590057fd66155a92c534201aff75d8f"
server_key_version: "encryptionkey@0"
I20260810 14:47:54.149346 10330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:54.174009 10330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:54.179867 10330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:54.180181 10330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:54.181564 10330 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:47:54.183296 10330 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:47:54.183422 10330 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:54.183522 10330 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:47:54.183596 10330 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:54.219342 10330 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:32857
I20260810 14:47:54.219434 10492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:32857 every 8 connection(s)
I20260810 14:47:54.220918 10330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:47:54.223841 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 10330
I20260810 14:47:54.224685 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:45417
--local_ip_for_outbound_sockets=127.24.215.130
--tserver_master_addrs=127.24.215.190:35483
--webserver_port=46843
--webserver_interface=127.24.215.130
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:47:54 dist-test-slave-vkjp krb5kdc[9615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373274, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:47:54.249840 10496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:54.224202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:46429)
0810 14:47:54.225506 (+ 1304us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:54.225553 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:54.227401 (+ 1848us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:54.229103 (+ 1702us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:54.229110 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:54.229664 (+ 554us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:54.230208 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:54.230226 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:54.233025 (+ 2799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:54.233029 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:54.234323 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:54.234338 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:54.235675 (+ 1337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:54.235685 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:54.236056 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:54.236881 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:54.236930 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:54.240245 (+ 3315us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:54.245337 (+ 5092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:54.245346 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:54.245360 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:54.245835 (+ 475us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:54.246380 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:54.246384 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:54.246386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:54.246572 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:54.247696 (+ 1124us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:54.247702 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:54.248022 (+ 320us) client_negotiation.cc:769] Sending connection context
0810 14:47:54.248270 (+ 248us) client_negotiation.cc:240] Negotiation successful
0810 14:47:54.248695 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1016,"thread_start_us":794,"threads_started":1}
I20260810 14:47:54.250365 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:54.224052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:46429 (local address 127.24.215.190:35483)
0810 14:47:54.224851 (+ 799us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:54.224884 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:54.225723 (+ 839us) server_negotiation.cc:407] Connection header received
0810 14:47:54.228009 (+ 2286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:54.228031 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:54.228540 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:54.228961 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:54.230513 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:54.232820 (+ 2307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:54.234502 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:54.235393 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:54.235546 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:54.240618 (+ 5072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:54.240633 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:54.240659 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:54.240725 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:54.245143 (+ 4418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:54.245992 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:54.246004 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:54.246015 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:54.246249 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:54.246686 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:54.246694 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:54.246700 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:54.247551 (+ 851us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:54.247748 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:54.248540 (+ 792us) server_negotiation.cc:299] Negotiation successful
0810 14:47:54.249041 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":509,"thread_start_us":336,"threads_started":1}
I20260810 14:47:54.253304 10493 heartbeater.cc:347] Connected to a master server at 127.24.215.190:35483
I20260810 14:47:54.253691 10493 heartbeater.cc:464] Registering TS with master...
I20260810 14:47:54.254868 10493 heartbeater.cc:511] Master 127.24.215.190:35483 requested a full tablet report, sending...
I20260810 14:47:54.257392 10288 ts_manager.cc:194] Registered new tserver with Master: 1cfe7b76354a4695b8a966262e90ba63 (127.24.215.129:32857)
I20260810 14:47:54.258890 10288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:46429
W20260810 14:47:54.574865 10497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:54.575181 10497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:54.575362 10497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:54.583135 10497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:54.583297 10497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:54.583487 10497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:47:54.593604 10497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:45417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=46843
--tserver_master_addrs=127.24.215.190:35483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:54.594982 10497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:54.596468 10497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:54.610487 10505 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:54.610558 10509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:54.610589 10506 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:54.611454 10497 server_base.cc:1034] running on GCE node
I20260810 14:47:54.612143 10497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:54.613124 10497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:54.614435 10497 hybrid_clock.cc:648] HybridClock initialized: now 1786373274614331 us; error 42 us; skew 500 ppm
I20260810 14:47:54.614801 10497 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:54 dist-test-slave-vkjp krb5kdc[9615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373274, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:54.618100 10497 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:47:54.619727 10497 webserver.cc:460] Webserver started at http://127.24.215.130:46843/ using document root <none> and password file <none>
I20260810 14:47:54.620453 10497 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:54.620618 10497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:54.626266 10497 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260810 14:47:54.629159 10517 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:54.630785 10497 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:47:54.630973 10497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "7770247d0e9043c1aff1592d4d084f4d"
format_stamp: "Formatted at 2026-08-10 14:47:51 on dist-test-slave-vkjp"
server_key: "e2fdbda65d52d001cdb3dfbf41fdf6bf"
server_key_iv: "c92460cccee1a4086388838761430836"
server_key_version: "encryptionkey@0"
I20260810 14:47:54.632401 10497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:54.649839 10497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:54.654083 10497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:54.654433 10497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:54.655844 10497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:47:54.657246 10497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:47:54.657392 10497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:54.657509 10497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:47:54.657574 10497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:54.692202 10497 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:45417
I20260810 14:47:54.692408 10647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:45417 every 8 connection(s)
I20260810 14:47:54.693784 10497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:47:54.698812 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 10497
I20260810 14:47:54.699590 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:34537
--local_ip_for_outbound_sockets=127.24.215.131
--tserver_master_addrs=127.24.215.190:35483
--webserver_port=38219
--webserver_interface=127.24.215.131
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:47:54 dist-test-slave-vkjp krb5kdc[9615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373274, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:47:54.715735 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:54.696567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:37335 (local address 127.24.215.190:35483)
0810 14:47:54.696751 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:54.696756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:54.698069 (+ 1313us) server_negotiation.cc:407] Connection header received
0810 14:47:54.700201 (+ 2132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:54.700206 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:54.700327 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:54.700454 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:54.702063 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:54.702787 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:54.704425 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:54.704891 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:54.705030 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:54.708926 (+ 3896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:54.708933 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:54.708938 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:54.708983 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:54.712485 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:54.713244 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:54.713257 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:54.713261 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:54.713424 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:54.713872 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:54.713877 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:54.713881 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:54.714176 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:54.714372 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:54.715116 (+ 744us) server_negotiation.cc:299] Negotiation successful
0810 14:47:54.715385 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 14:47:54.716784 10651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:54.697139 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:37335)
0810 14:47:54.697907 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:54.697944 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:54.699913 (+ 1969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:54.700709 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:54.700719 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:54.701312 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:54.701885 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:54.701903 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:54.702967 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:54.702970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:54.704261 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:54.704274 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:54.705131 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:54.705138 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:54.705390 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:54.706095 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:54.706117 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:54.708698 (+ 2581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:54.712641 (+ 3943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:54.712650 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:54.712667 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:54.713108 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:54.713542 (+ 434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:54.713547 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:54.713551 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:54.713746 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:54.714312 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:54.714323 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:54.714687 (+ 364us) client_negotiation.cc:769] Sending connection context
0810 14:47:54.714960 (+ 273us) client_negotiation.cc:240] Negotiation successful
0810 14:47:54.715532 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"thread_start_us":314,"threads_started":1}
I20260810 14:47:54.718330 10649 heartbeater.cc:347] Connected to a master server at 127.24.215.190:35483
I20260810 14:47:54.718750 10649 heartbeater.cc:464] Registering TS with master...
I20260810 14:47:54.719571 10649 heartbeater.cc:511] Master 127.24.215.190:35483 requested a full tablet report, sending...
I20260810 14:47:54.721079 10288 ts_manager.cc:194] Registered new tserver with Master: 7770247d0e9043c1aff1592d4d084f4d (127.24.215.130:45417)
I20260810 14:47:54.721915 10288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:37335
W20260810 14:47:55.054502 10652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:55.054800 10652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:55.054899 10652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:55.062837 10652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:55.062964 10652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:55.063231 10652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:47:55.074939 10652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:34537
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=38219
--tserver_master_addrs=127.24.215.190:35483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:55.076428 10652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:55.077960 10652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:55.092677 10660 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:55.092656 10662 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:55.092721 10658 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:55.093408 10652 server_base.cc:1034] running on GCE node
I20260810 14:47:55.094106 10652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:55.095052 10652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:55.096338 10652 hybrid_clock.cc:648] HybridClock initialized: now 1786373275096274 us; error 52 us; skew 500 ppm
I20260810 14:47:55.096658 10652 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:55 dist-test-slave-vkjp krb5kdc[9615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373275, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:55.105114 10652 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:47:55.106779 10652 webserver.cc:460] Webserver started at http://127.24.215.131:38219/ using document root <none> and password file <none>
I20260810 14:47:55.107579 10652 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:55.107755 10652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:55.113619 10652 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 14:47:55.116555 10669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:55.117890 10652 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:47:55.118065 10652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "5f800c59113d401dbece99261430a69e"
format_stamp: "Formatted at 2026-08-10 14:47:51 on dist-test-slave-vkjp"
server_key: "13146e610d1be4baf699389e3ee5443e"
server_key_iv: "b16cea9b57178707ab98b0b23f777a55"
server_key_version: "encryptionkey@0"
I20260810 14:47:55.119047 10652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:55.139736 10652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:55.143841 10652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:55.144127 10652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:55.145130 10652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:47:55.146461 10652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:47:55.146548 10652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:55.146703 10652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:47:55.146801 10652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:55.179075 10652 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:34537
I20260810 14:47:55.179168 10797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:34537 every 8 connection(s)
I20260810 14:47:55.180572 10652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:47:55.184504 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 10652
Aug 10 14:47:55 dist-test-slave-vkjp krb5kdc[9615](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373275, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:47:55.203711 10801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.185518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39036 (local address 127.24.215.190:35483)
0810 14:47:55.187792 (+ 2274us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:55.187796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:55.187811 (+ 15us) server_negotiation.cc:407] Connection header received
0810 14:47:55.187852 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:55.187855 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:55.187901 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:55.187983 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:55.189983 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.191145 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.193114 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.193689 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.193825 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.197421 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:55.197430 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:55.197436 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:55.197512 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:55.200485 (+ 2973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:55.201472 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:55.201476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:55.201479 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:55.201643 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:55.202076 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:55.202082 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:55.202085 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:55.202392 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:55.202665 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:55.203062 (+ 397us) server_negotiation.cc:299] Negotiation successful
0810 14:47:55.203324 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":2185,"thread_start_us":2055,"threads_started":1}
I20260810 14:47:55.204918 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.183298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:38545 (local address 127.24.215.190:35483)
0810 14:47:55.183500 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:55.183504 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:55.184620 (+ 1116us) server_negotiation.cc:407] Connection header received
0810 14:47:55.187428 (+ 2808us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:55.187434 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:55.187527 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:55.187664 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:55.189766 (+ 2102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.191043 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.193382 (+ 2339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.194053 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.194150 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.198285 (+ 4135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:55.198292 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:55.198298 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:55.198354 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:55.201556 (+ 3202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:55.202311 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:55.202319 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:55.202325 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:55.202450 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:55.202894 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:55.202900 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:55.202904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:55.203259 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:55.203412 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:55.204328 (+ 916us) server_negotiation.cc:299] Negotiation successful
0810 14:47:55.204526 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":56}
I20260810 14:47:55.206125 10800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.183739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:38545)
0810 14:47:55.184467 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:55.184504 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:55.187140 (+ 2636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:55.187904 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:55.187913 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:55.188697 (+ 784us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:47:55.189570 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:55.189598 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.191259 (+ 1661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.191265 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:55.193256 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:55.193268 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.194301 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.194310 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:55.194593 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.195246 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:47:55.195288 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:47:55.198075 (+ 2787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:47:55.201714 (+ 3639us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:55.201723 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:55.201742 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:55.202187 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:55.202571 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:47:55.202576 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:47:55.202580 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:47:55.202774 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:47:55.203392 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:47:55.203400 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:47:55.203861 (+ 461us) client_negotiation.cc:769] Sending connection context
0810 14:47:55.204222 (+ 361us) client_negotiation.cc:240] Negotiation successful
0810 14:47:55.204799 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":310,"threads_started":1}
I20260810 14:47:55.207617 10798 heartbeater.cc:347] Connected to a master server at 127.24.215.190:35483
I20260810 14:47:55.207933 10798 heartbeater.cc:464] Registering TS with master...
I20260810 14:47:55.208693 10798 heartbeater.cc:511] Master 127.24.215.190:35483 requested a full tablet report, sending...
I20260810 14:47:55.210098 10288 ts_manager.cc:194] Registered new tserver with Master: 5f800c59113d401dbece99261430a69e (127.24.215.131:34537)
I20260810 14:47:55.210918 10288 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:38545
I20260810 14:47:55.219245 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:47:55.226745 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.220290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39038 (local address 127.24.215.190:35483)
0810 14:47:55.220493 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:55.220500 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:55.220923 (+ 423us) server_negotiation.cc:407] Connection header received
0810 14:47:55.221206 (+ 283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:55.221210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:55.221353 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:55.221513 (+ 160us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:47:55.221957 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.222924 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.224503 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.224955 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.225098 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.225660 (+ 562us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:47:55.226131 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:47:55.226290 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:55.226391 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 14:47:55.226486 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:47:55.227768 10288 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:39038:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:47:55.235306 10288 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:47:55.262578 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.247471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54156 (local address 127.24.215.129:32857)
0810 14:47:55.248200 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:55.248210 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:55.249301 (+ 1091us) server_negotiation.cc:407] Connection header received
0810 14:47:55.249789 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:55.249802 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:55.250139 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:55.250334 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:47:55.251660 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.253820 (+ 2160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.258060 (+ 4240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.260263 (+ 2203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.260516 (+ 253us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.260773 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:55.261789 (+ 1016us) server_negotiation.cc:299] Negotiation successful
0810 14:47:55.262070 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":392,"threads_started":1}
I20260810 14:47:55.262645 10493 heartbeater.cc:503] Master 127.24.215.190:35483 was elected leader, sending a full tablet report...
I20260810 14:47:55.263643 10810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.247338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:32857 (local address 127.0.0.1:54156)
0810 14:47:55.249098 (+ 1760us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:55.249158 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:55.249497 (+ 339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:55.250744 (+ 1247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:55.250755 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:55.250799 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:47:55.251377 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:55.251395 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.254231 (+ 2836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.254239 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:55.257754 (+ 3515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:55.257781 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.261081 (+ 3300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.261091 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:55.261413 (+ 322us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.261487 (+ 74us) client_negotiation.cc:769] Sending connection context
0810 14:47:55.262858 (+ 1371us) client_negotiation.cc:240] Negotiation successful
0810 14:47:55.263116 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1411,"thread_start_us":1260,"threads_started":1}
I20260810 14:47:55.270205 10425 tablet_service.cc:1467] Processing CreateTablet for tablet bafcd619e45e4cdf8e49ab2116115e2e (DEFAULT_TABLE table=test-table [id=aed789cb42344fd1979792890dbe9352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:47:55.271806 10425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bafcd619e45e4cdf8e49ab2116115e2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:47:55.272442 10810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.248188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:34537 (local address 127.0.0.1:50370)
0810 14:47:55.263919 (+ 15731us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:55.263946 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:55.264113 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:55.265526 (+ 1413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:55.265533 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:55.265566 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:47:55.265797 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:55.265834 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.268021 (+ 2187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.268027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:55.269520 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:55.269535 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.271614 (+ 2079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.271622 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:55.271812 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.271855 (+ 43us) client_negotiation.cc:769] Sending connection context
0810 14:47:55.272018 (+ 163us) client_negotiation.cc:240] Negotiation successful
0810 14:47:55.272176 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15625,"thread_start_us":16976,"threads_started":1}
I20260810 14:47:55.272792 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.248570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50370 (local address 127.24.215.131:34537)
0810 14:47:55.264781 (+ 16211us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:55.264794 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:55.264832 (+ 38us) server_negotiation.cc:407] Connection header received
0810 14:47:55.264946 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:55.264956 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:55.265270 (+ 314us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:55.265475 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:47:55.266042 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.267861 (+ 1819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.269731 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.271439 (+ 1708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.271632 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.271839 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:55.272080 (+ 241us) server_negotiation.cc:299] Negotiation successful
0810 14:47:55.272261 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15989,"thread_start_us":16651,"threads_started":1}
I20260810 14:47:55.273643 10814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.250237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:45417 (local address 127.0.0.1:56470)
0810 14:47:55.264934 (+ 14697us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:55.264952 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:55.265086 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:55.266378 (+ 1292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:55.266382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:55.266407 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:47:55.266770 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:55.266782 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.268328 (+ 1546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.268333 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:55.270427 (+ 2094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:55.270442 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.272570 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.272576 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:55.272750 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.272795 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 14:47:55.272892 (+ 97us) client_negotiation.cc:240] Negotiation successful
0810 14:47:55.273288 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14561,"thread_start_us":14767,"threads_started":1}
I20260810 14:47:55.273650 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.249873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56470 (local address 127.24.215.130:45417)
0810 14:47:55.265578 (+ 15705us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:55.265588 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:55.265619 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:47:55.265722 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:55.265728 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:55.265948 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:55.266148 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:47:55.266983 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.268171 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.270596 (+ 2425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.272095 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.272254 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.272420 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:55.273157 (+ 737us) server_negotiation.cc:299] Negotiation successful
0810 14:47:55.273320 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15516,"thread_start_us":15630,"threads_started":1}
I20260810 14:47:55.278100 10575 tablet_service.cc:1467] Processing CreateTablet for tablet bafcd619e45e4cdf8e49ab2116115e2e (DEFAULT_TABLE table=test-table [id=aed789cb42344fd1979792890dbe9352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:47:55.278100 10724 tablet_service.cc:1467] Processing CreateTablet for tablet bafcd619e45e4cdf8e49ab2116115e2e (DEFAULT_TABLE table=test-table [id=aed789cb42344fd1979792890dbe9352]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:47:55.279680 10575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bafcd619e45e4cdf8e49ab2116115e2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:47:55.279666 10724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bafcd619e45e4cdf8e49ab2116115e2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:47:55.286343 10818 tablet_bootstrap.cc:492] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63: Bootstrap starting.
I20260810 14:47:55.290133 10818 tablet_bootstrap.cc:654] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63: Neither blocks nor log segments found. Creating new log.
I20260810 14:47:55.291365 10818 log.cc:824] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63: Log is configured to *not* fsync() on all Append() calls
I20260810 14:47:55.294080 10818 tablet_bootstrap.cc:492] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63: No bootstrap required, opened a new log
I20260810 14:47:55.294385 10818 ts_tablet_manager.cc:1397] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 14:47:55.296060 10820 tablet_bootstrap.cc:492] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e: Bootstrap starting.
I20260810 14:47:55.297837 10821 tablet_bootstrap.cc:492] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d: Bootstrap starting.
I20260810 14:47:55.301045 10821 tablet_bootstrap.cc:654] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d: Neither blocks nor log segments found. Creating new log.
I20260810 14:47:55.301216 10820 tablet_bootstrap.cc:654] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e: Neither blocks nor log segments found. Creating new log.
I20260810 14:47:55.302217 10821 log.cc:824] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:47:55.302430 10820 log.cc:824] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:47:55.305001 10821 tablet_bootstrap.cc:492] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d: No bootstrap required, opened a new log
I20260810 14:47:55.305315 10821 ts_tablet_manager.cc:1397] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260810 14:47:55.305686 10818 raft_consensus.cc:348] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } } peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } }
I20260810 14:47:55.306211 10818 raft_consensus.cc:374] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:47:55.306324 10818 raft_consensus.cc:729] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cfe7b76354a4695b8a966262e90ba63, State: Initialized, Role: FOLLOWER
I20260810 14:47:55.307132 10818 consensus_queue.cc:260] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } } peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } }
I20260810 14:47:55.308616 10820 tablet_bootstrap.cc:492] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e: No bootstrap required, opened a new log
I20260810 14:47:55.308908 10820 ts_tablet_manager.cc:1397] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.008s
I20260810 14:47:55.309098 10818 ts_tablet_manager.cc:1428] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63: Time spent starting tablet: real 0.015s user 0.012s sys 0.004s
I20260810 14:47:55.314338 10821 raft_consensus.cc:348] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } } peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } }
I20260810 14:47:55.314793 10821 raft_consensus.cc:374] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:47:55.314911 10821 raft_consensus.cc:729] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7770247d0e9043c1aff1592d4d084f4d, State: Initialized, Role: FOLLOWER
I20260810 14:47:55.315528 10821 consensus_queue.cc:260] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } } peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } }
I20260810 14:47:55.316663 10649 heartbeater.cc:503] Master 127.24.215.190:35483 was elected leader, sending a full tablet report...
I20260810 14:47:55.317373 10821 ts_tablet_manager.cc:1428] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d: Time spent starting tablet: real 0.012s user 0.013s sys 0.000s
I20260810 14:47:55.318845 10820 raft_consensus.cc:348] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } } peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } }
I20260810 14:47:55.319396 10820 raft_consensus.cc:374] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:47:55.319540 10820 raft_consensus.cc:729] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f800c59113d401dbece99261430a69e, State: Initialized, Role: FOLLOWER
I20260810 14:47:55.320183 10820 consensus_queue.cc:260] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } } peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } }
I20260810 14:47:55.321405 10798 heartbeater.cc:503] Master 127.24.215.190:35483 was elected leader, sending a full tablet report...
I20260810 14:47:55.322208 10820 ts_tablet_manager.cc:1428] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260810 14:47:55.414376 10827 raft_consensus.cc:482] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:47:55.414731 10827 raft_consensus.cc:504] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } } peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } }
I20260810 14:47:55.416700 10827 leader_election.cc:288] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cfe7b76354a4695b8a966262e90ba63 (127.24.215.129:32857), 5f800c59113d401dbece99261430a69e (127.24.215.131:34537)
I20260810 14:47:55.424508 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.417281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:55365 (local address 127.24.215.129:32857)
0810 14:47:55.417497 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:55.417526 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:55.417996 (+ 470us) server_negotiation.cc:407] Connection header received
0810 14:47:55.418258 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:55.418262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:55.418357 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:55.418447 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:47:55.418987 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.419789 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.422389 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.423531 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.423716 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.423798 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:55.424106 (+ 308us) server_negotiation.cc:299] Negotiation successful
0810 14:47:55.424234 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260810 14:47:55.424791 10829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.417021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:32857 (local address 127.24.215.130:55365)
0810 14:47:55.417863 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:55.417886 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:55.418097 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:55.418467 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:55.418471 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:55.418497 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:47:55.418835 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:55.418844 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.419921 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.419928 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:55.422208 (+ 2280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:55.422221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.423764 (+ 1543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.423771 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:55.423919 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.423942 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 14:47:55.424074 (+ 132us) client_negotiation.cc:240] Negotiation successful
0810 14:47:55.424220 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"mutex_wait_us":385,"thread_start_us":537,"threads_started":1}
I20260810 14:47:55.424930 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.417373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:46047 (local address 127.24.215.131:34537)
0810 14:47:55.417631 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:55.417641 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:55.418358 (+ 717us) server_negotiation.cc:407] Connection header received
0810 14:47:55.418557 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:55.418561 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:55.418626 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:55.418738 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:47:55.419428 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.420216 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.422632 (+ 2416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.423869 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.424061 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.424151 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:55.424491 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 14:47:55.424626 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260810 14:47:55.425172 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.417228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:34537 (local address 127.24.215.130:46047)
0810 14:47:55.418234 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:55.418250 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:47:55.418379 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:47:55.418759 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:47:55.418763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:47:55.418808 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:47:55.419258 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:55.419271 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.420401 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.420407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:55.422455 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:47:55.422469 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.424114 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.424121 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:47:55.424315 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.424340 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 14:47:55.424462 (+ 122us) client_negotiation.cc:240] Negotiation successful
0810 14:47:55.424600 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"mutex_wait_us":26,"thread_start_us":623,"threads_started":1}
I20260810 14:47:55.426179 10446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bafcd619e45e4cdf8e49ab2116115e2e" candidate_uuid: "7770247d0e9043c1aff1592d4d084f4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cfe7b76354a4695b8a966262e90ba63" is_pre_election: true
I20260810 14:47:55.426306 10745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bafcd619e45e4cdf8e49ab2116115e2e" candidate_uuid: "7770247d0e9043c1aff1592d4d084f4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f800c59113d401dbece99261430a69e" is_pre_election: true
I20260810 14:47:55.426889 10745 raft_consensus.cc:2454] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7770247d0e9043c1aff1592d4d084f4d in term 0.
I20260810 14:47:55.426889 10446 raft_consensus.cc:2454] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7770247d0e9043c1aff1592d4d084f4d in term 0.
I20260810 14:47:55.427780 10521 leader_election.cc:302] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 1cfe7b76354a4695b8a966262e90ba63, 5f800c59113d401dbece99261430a69e, 7770247d0e9043c1aff1592d4d084f4d; no voters:
I20260810 14:47:55.428300 10827 raft_consensus.cc:2781] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:47:55.428465 10827 raft_consensus.cc:482] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:47:55.428598 10827 raft_consensus.cc:3037] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:47:55.430845 10827 raft_consensus.cc:504] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } } peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } }
I20260810 14:47:55.431566 10827 leader_election.cc:288] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [CANDIDATE]: Term 1 election: Requested vote from peers 1cfe7b76354a4695b8a966262e90ba63 (127.24.215.129:32857), 5f800c59113d401dbece99261430a69e (127.24.215.131:34537)
I20260810 14:47:55.432108 10745 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bafcd619e45e4cdf8e49ab2116115e2e" candidate_uuid: "7770247d0e9043c1aff1592d4d084f4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f800c59113d401dbece99261430a69e"
I20260810 14:47:55.432147 10446 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bafcd619e45e4cdf8e49ab2116115e2e" candidate_uuid: "7770247d0e9043c1aff1592d4d084f4d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cfe7b76354a4695b8a966262e90ba63"
W20260810 14:47:55.432260 10799 tablet.cc:2369] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:47:55.432376 10745 raft_consensus.cc:3037] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:47:55.432387 10446 raft_consensus.cc:3037] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:47:55.434847 10446 raft_consensus.cc:2454] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7770247d0e9043c1aff1592d4d084f4d in term 1.
I20260810 14:47:55.434839 10745 raft_consensus.cc:2454] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7770247d0e9043c1aff1592d4d084f4d in term 1.
I20260810 14:47:55.435343 10519 leader_election.cc:302] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f800c59113d401dbece99261430a69e, 7770247d0e9043c1aff1592d4d084f4d; no voters:
I20260810 14:47:55.435685 10827 raft_consensus.cc:2781] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:47:55.436189 10827 raft_consensus.cc:686] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [term 1 LEADER]: Becoming Leader. State: Replica: 7770247d0e9043c1aff1592d4d084f4d, State: Running, Role: LEADER
I20260810 14:47:55.436728 10827 consensus_queue.cc:237] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } } peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } }
I20260810 14:47:55.441638 10288 catalog_manager.cc:5495] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d reported cstate change: term changed from 0 to 1, leader changed from <none> to 7770247d0e9043c1aff1592d4d084f4d (127.24.215.130). New cstate: current_term: 1 leader_uuid: "7770247d0e9043c1aff1592d4d084f4d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7770247d0e9043c1aff1592d4d084f4d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 45417 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 } health_report { overall_health: UNKNOWN } } }
W20260810 14:47:55.445878 10650 tablet.cc:2369] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:47:55.470583 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.465434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56478 (local address 127.24.215.130:45417)
0810 14:47:55.465632 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:55.465637 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:55.465655 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:47:55.465776 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:55.465780 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:55.465857 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:55.465983 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:47:55.466358 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.467173 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.468524 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.468926 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.469014 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.469499 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:47:55.470004 (+ 505us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:47:55.470121 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:55.470222 (+ 101us) server_negotiation.cc:299] Negotiation successful
0810 14:47:55.470286 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260810 14:47:55.473402 10494 tablet.cc:2369] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:47:55.479429 10745 raft_consensus.cc:1261] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e [term 1 FOLLOWER]: Refusing update from remote peer 7770247d0e9043c1aff1592d4d084f4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:47:55.479429 10446 raft_consensus.cc:1261] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63 [term 1 FOLLOWER]: Refusing update from remote peer 7770247d0e9043c1aff1592d4d084f4d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:47:55.480612 10831 consensus_queue.cc:1035] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cfe7b76354a4695b8a966262e90ba63" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32857 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:47:55.481042 10827 consensus_queue.cc:1035] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f800c59113d401dbece99261430a69e" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34537 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:47:55.508749 10287 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:39038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:47:55.509093 10287 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:39038:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:47:55.512948 10287 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4: Sending DeleteTablet for 3 replicas of tablet bafcd619e45e4cdf8e49ab2116115e2e
I20260810 14:47:55.514384 10575 tablet_service.cc:1514] Processing DeleteTablet for tablet bafcd619e45e4cdf8e49ab2116115e2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:47:55 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:56470
I20260810 14:47:55.514485 10724 tablet_service.cc:1514] Processing DeleteTablet for tablet bafcd619e45e4cdf8e49ab2116115e2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:47:55 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:50370
I20260810 14:47:55.514729 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 10330
I20260810 14:47:55.515213 10425 tablet_service.cc:1514] Processing DeleteTablet for tablet bafcd619e45e4cdf8e49ab2116115e2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:47:55 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:54156
I20260810 14:47:55.516782 10847 tablet_replica.cc:330] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d: stopping tablet replica
I20260810 14:47:55.517217 10847 raft_consensus.cc:2229] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:47:55.517900 10847 raft_consensus.cc:2258] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:47:55.519302 10846 tablet_replica.cc:330] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e: stopping tablet replica
I20260810 14:47:55.519306 10848 tablet_replica.cc:330] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63: stopping tablet replica
I20260810 14:47:55.519737 10848 raft_consensus.cc:2229] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:47:55.519740 10846 raft_consensus.cc:2229] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:47:55.520269 10846 raft_consensus.cc:2258] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:47:55.520676 10848 raft_consensus.cc:2258] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:47:55.521227 10847 ts_tablet_manager.cc:1905] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:47:55.522908 10848 ts_tablet_manager.cc:1905] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:47:55.529052 10847 ts_tablet_manager.cc:1918] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:47:55.529242 10847 log.cc:1196] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/bafcd619e45e4cdf8e49ab2116115e2e
I20260810 14:47:55.529866 10847 ts_tablet_manager.cc:1939] T bafcd619e45e4cdf8e49ab2116115e2e P 7770247d0e9043c1aff1592d4d084f4d: Deleting consensus metadata
I20260810 14:47:55.530712 10848 ts_tablet_manager.cc:1918] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:47:55.530982 10848 log.cc:1196] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/bafcd619e45e4cdf8e49ab2116115e2e
I20260810 14:47:55.531891 10848 ts_tablet_manager.cc:1939] T bafcd619e45e4cdf8e49ab2116115e2e P 1cfe7b76354a4695b8a966262e90ba63: Deleting consensus metadata
I20260810 14:47:55.533143 10846 ts_tablet_manager.cc:1905] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:47:55.533941 10272 catalog_manager.cc:4841] TS 1cfe7b76354a4695b8a966262e90ba63 (127.24.215.129:32857): tablet bafcd619e45e4cdf8e49ab2116115e2e (table test-table [id=aed789cb42344fd1979792890dbe9352]) successfully deleted
I20260810 14:47:55.535111 10272 catalog_manager.cc:4841] TS 7770247d0e9043c1aff1592d4d084f4d (127.24.215.130:45417): tablet bafcd619e45e4cdf8e49ab2116115e2e (table test-table [id=aed789cb42344fd1979792890dbe9352]) successfully deleted
I20260810 14:47:55.536761 10812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.515756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54162 (local address 127.24.215.129:32857)
0810 14:47:55.515940 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:55.515947 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:55.516010 (+ 63us) server_negotiation.cc:407] Connection header received
0810 14:47:55.516217 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:55.516223 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:55.516311 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:55.516415 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:55.519020 (+ 2605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.521712 (+ 2692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.523907 (+ 2195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.524565 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.524703 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.528632 (+ 3929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:55.528647 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:55.528670 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:55.528713 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:55.532964 (+ 4251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:55.534028 (+ 1064us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:55.534036 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:55.534039 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:55.534206 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:55.534712 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:55.534716 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:55.534719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:55.535238 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:55.535569 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:55.536033 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 14:47:55.536257 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:47:55.537636 10488 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:55.541179 10846 ts_tablet_manager.cc:1918] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:47:55.541359 10846 log.cc:1196] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/bafcd619e45e4cdf8e49ab2116115e2e
I20260810 14:47:55.541940 10846 ts_tablet_manager.cc:1939] T bafcd619e45e4cdf8e49ab2116115e2e P 5f800c59113d401dbece99261430a69e: Deleting consensus metadata
I20260810 14:47:55.543888 10271 catalog_manager.cc:4841] TS 5f800c59113d401dbece99261430a69e (127.24.215.131:34537): tablet bafcd619e45e4cdf8e49ab2116115e2e (table test-table [id=aed789cb42344fd1979792890dbe9352]) successfully deleted
I20260810 14:47:55.631554 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 10330
I20260810 14:47:55.659178 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 10497
I20260810 14:47:55.675693 10816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.660060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56486 (local address 127.24.215.130:45417)
0810 14:47:55.660240 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:55.660245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:55.660262 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:47:55.660448 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:55.660452 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:55.660530 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:55.660616 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:55.662411 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.663331 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.665335 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.665780 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.665962 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.669427 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:55.669440 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:55.669456 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:55.669498 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:55.672720 (+ 3222us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:55.673576 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:55.673583 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:55.673586 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:55.673719 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:55.674150 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:55.674154 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:55.674156 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:55.674518 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:55.674793 (+ 275us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:55.675181 (+ 388us) server_negotiation.cc:299] Negotiation successful
0810 14:47:55.675358 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 14:47:55.676550 10642 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:55.791337 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 10497
I20260810 14:47:55.814457 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 10652
I20260810 14:47:55.828820 10815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:55.815262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50374 (local address 127.24.215.131:34537)
0810 14:47:55.815476 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 14:47:55.815483 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:47:55.815594 (+ 111us) server_negotiation.cc:407] Connection header received
0810 14:47:55.815829 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:47:55.815834 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:47:55.815900 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:47:55.816006 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:47:55.817511 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.818249 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.819725 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:47:55.820149 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:47:55.820284 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:47:55.822956 (+ 2672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:47:55.822971 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:47:55.822987 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:47:55.823026 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:47:55.825726 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:55.826595 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:55.826610 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:55.826612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:55.826769 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:47:55.827214 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:47:55.827220 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:47:55.827223 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:47:55.827587 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:47:55.827922 (+ 335us) server_negotiation.cc:1035] Waiting for connection context
0810 14:47:55.828238 (+ 316us) server_negotiation.cc:299] Negotiation successful
0810 14:47:55.828459 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:47:55.829488 10793 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:55.920444 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 10652
I20260810 14:47:55.942044 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 10252
I20260810 14:47:55.943372 10321 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:47:56.058405 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 10252
I20260810 14:47:56.089133 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:35483
--webserver_interface=127.24.215.190
--webserver_port=45995
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:35483
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:47:56.426580 10855 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:56.426855 10855 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:56.427026 10855 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:56.434736 10855 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:47:56.434891 10855 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:56.435042 10855 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:56.435192 10855 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:47:56.435307 10855 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:47:56.445957 10855 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:35483
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:35483
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=45995
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:56.447461 10855 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:56.448822 10855 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:56.461313 10861 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:56.461408 10862 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:56.461472 10864 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:56.462123 10855 server_base.cc:1034] running on GCE node
I20260810 14:47:56.462975 10855 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:56.464392 10855 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:56.465644 10855 hybrid_clock.cc:648] HybridClock initialized: now 1786373276465591 us; error 37 us; skew 500 ppm
I20260810 14:47:56.466018 10855 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:56 dist-test-slave-vkjp krb5kdc[9615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373276, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:56.469180 10855 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:47:56.470695 10855 webserver.cc:460] Webserver started at http://127.24.215.190:45995/ using document root <none> and password file <none>
I20260810 14:47:56.471453 10855 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:56.471612 10855 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:56.477008 10855 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 14:47:56.479935 10873 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:56.481230 10855 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:47:56.481455 10855 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "8ba6f5e397d141498c43714f9744daf4"
format_stamp: "Formatted at 2026-08-10 14:47:50 on dist-test-slave-vkjp"
server_key: "ef8dee847a0a00e82eaed55f29b61799"
server_key_iv: "a676982ec185e760a5dbd51e6a80fc14"
server_key_version: "encryptionkey@0"
I20260810 14:47:56.482455 10855 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:56.503394 10855 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:56.507383 10855 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:56.507663 10855 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:56.526728 10855 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:35483
I20260810 14:47:56.526747 10937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:35483 every 8 connection(s)
I20260810 14:47:56.528338 10855 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:47:56.535228 10938 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:47:56.535964 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 10855
I20260810 14:47:56.536820 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:32857
--local_ip_for_outbound_sockets=127.24.215.129
--tserver_master_addrs=127.24.215.190:35483
--webserver_port=35093
--webserver_interface=127.24.215.129
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:47:56.538748 10938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4: Bootstrap starting.
I20260810 14:47:56.563184 10938 log.cc:824] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4: Log is configured to *not* fsync() on all Append() calls
I20260810 14:47:56.587653 10938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:47:56.588336 10938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4: Bootstrap complete.
I20260810 14:47:56.602495 10938 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } }
I20260810 14:47:56.603832 10938 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ba6f5e397d141498c43714f9744daf4, State: Initialized, Role: FOLLOWER
I20260810 14:47:56.604585 10938 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } }
I20260810 14:47:56.604867 10938 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:47:56.604997 10938 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:47:56.605189 10938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 2 FOLLOWER]: Advancing to term 3
I20260810 14:47:56.611877 10938 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } }
I20260810 14:47:56.612527 10938 leader_election.cc:302] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8ba6f5e397d141498c43714f9744daf4; no voters:
I20260810 14:47:56.613232 10938 leader_election.cc:288] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 14:47:56.613392 10942 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 14:47:56.614768 10942 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [term 3 LEADER]: Becoming Leader. State: Replica: 8ba6f5e397d141498c43714f9744daf4, State: Running, Role: LEADER
I20260810 14:47:56.615449 10938 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:47:56.615481 10942 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } }
I20260810 14:47:56.617858 10944 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8ba6f5e397d141498c43714f9744daf4. Latest consensus state: current_term: 3 leader_uuid: "8ba6f5e397d141498c43714f9744daf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } } }
I20260810 14:47:56.618257 10944 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [sys.catalog]: This master's current role is: LEADER
I20260810 14:47:56.618866 10943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "8ba6f5e397d141498c43714f9744daf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8ba6f5e397d141498c43714f9744daf4" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 35483 } } }
I20260810 14:47:56.619199 10948 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:47:56.619236 10943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8ba6f5e397d141498c43714f9744daf4 [sys.catalog]: This master's current role is: LEADER
I20260810 14:47:56.637015 10948 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:47:56.639057 10948 catalog_manager.cc:1259] Loaded cluster ID: 2a22cd9fac2f4587b8b4354859ac4d89
I20260810 14:47:56.639304 10948 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260810 14:47:56.642226 10948 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1a0cfcb9ad google::LogMessage::Fail() at ??:0
@ 0x7f1a0cfce88a google::LogMessage::SendToLog() at ??:0
@ 0x7f1a0cfcb4de google::LogMessage::Flush() at ??:0
@ 0x7f1a0cfcca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1a229fe512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f1a229fb976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1a22a83610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f1a22a83439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f1a0fc17345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1a0fc1a91c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f1a0fc1a799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f1a0fbf5eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1a12ecf6db start_thread at ??:0
@ 0x7f1a0be6561f clone at ??:0
W20260810 14:47:56.977265 10940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:56.977550 10940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:56.977882 10940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:56.985361 10940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:56.985496 10940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:56.985698 10940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:47:56.995559 10940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:32857
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=35093
--tserver_master_addrs=127.24.215.190:35483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:56.996836 10940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:56.998215 10940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:57.011752 10985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:57.011802 10984 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:57.011778 10988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:57.012708 10940 server_base.cc:1034] running on GCE node
I20260810 14:47:57.013330 10940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:57.014303 10940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:57.015537 10940 hybrid_clock.cc:648] HybridClock initialized: now 1786373277015482 us; error 53 us; skew 500 ppm
I20260810 14:47:57.015877 10940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:57 dist-test-slave-vkjp krb5kdc[9615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373277, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:57.019025 10940 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:47:57.020555 10940 webserver.cc:460] Webserver started at http://127.24.215.129:35093/ using document root <none> and password file <none>
I20260810 14:47:57.021212 10940 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:57.021317 10940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:57.026651 10940 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:47:57.029654 10997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:57.030977 10940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260810 14:47:57.031210 10940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "1cfe7b76354a4695b8a966262e90ba63"
format_stamp: "Formatted at 2026-08-10 14:47:50 on dist-test-slave-vkjp"
server_key: "0cb3cbbf06983095784cacfef4d2917c"
server_key_iv: "5590057fd66155a92c534201aff75d8f"
server_key_version: "encryptionkey@0"
I20260810 14:47:57.032192 10940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:57.052245 10940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:57.056183 10940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:57.056416 10940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:57.057380 10940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:47:57.058739 10940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:47:57.058827 10940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:57.058914 10940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:47:57.058972 10940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:57.095006 10940 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:32857
I20260810 14:47:57.095062 11126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:32857 every 8 connection(s)
I20260810 14:47:57.096566 10940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:47:57.100251 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 10940
I20260810 14:47:57.100903 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:45417
--local_ip_for_outbound_sockets=127.24.215.130
--tserver_master_addrs=127.24.215.190:35483
--webserver_port=46843
--webserver_interface=127.24.215.130
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:47:57.101657 11129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:57.099528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:44563)
0810 14:47:57.100408 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:57.100688 (+ 280us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":364,"threads_started":1}
W20260810 14:47:57.102279 11127 heartbeater.cc:644] Failed to heartbeat to 127.24.215.190:35483 (0 consecutive failures): Network error: Failed to ping master at 127.24.215.190:35483: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
I20260810 14:47:57.103233 11129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:57.102876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:46021)
0810 14:47:57.103016 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:57.103121 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 14:47:57.104067 11129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:57.103739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:34865)
0810 14:47:57.103890 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:57.103963 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 14:47:57.104276 11127 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:47:57.441458 11130 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:57.441773 11130 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:57.441874 11130 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:57.449250 11130 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:57.449373 11130 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:57.449487 11130 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:47:57.459378 11130 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:45417
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=46843
--tserver_master_addrs=127.24.215.190:35483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:57.460690 11130 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:57.462087 11130 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:57.475646 11139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:57.475687 11135 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:57.475610 11136 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:57.476581 11130 server_base.cc:1034] running on GCE node
I20260810 14:47:57.477213 11130 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:57.478202 11130 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:57.479422 11130 hybrid_clock.cc:648] HybridClock initialized: now 1786373277479357 us; error 58 us; skew 500 ppm
I20260810 14:47:57.479776 11130 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:57 dist-test-slave-vkjp krb5kdc[9615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373277, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:57.483075 11130 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:47:57.484612 11130 webserver.cc:460] Webserver started at http://127.24.215.130:46843/ using document root <none> and password file <none>
I20260810 14:47:57.485333 11130 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:57.485489 11130 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:57.491186 11130 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260810 14:47:57.494315 11148 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:57.495718 11130 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:47:57.495903 11130 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "7770247d0e9043c1aff1592d4d084f4d"
format_stamp: "Formatted at 2026-08-10 14:47:51 on dist-test-slave-vkjp"
server_key: "e2fdbda65d52d001cdb3dfbf41fdf6bf"
server_key_iv: "c92460cccee1a4086388838761430836"
server_key_version: "encryptionkey@0"
I20260810 14:47:57.496935 11130 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:57.508896 11130 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:57.513293 11130 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:57.513586 11130 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:57.514572 11130 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:47:57.515995 11130 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:47:57.516115 11130 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:57.516240 11130 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:47:57.516328 11130 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:57.550575 11130 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:45417
I20260810 14:47:57.550665 11272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:45417 every 8 connection(s)
I20260810 14:47:57.552150 11130 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:47:57.557327 11275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:57.555425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:40777)
0810 14:47:57.556211 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:57.556460 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":313,"threads_started":1}
W20260810 14:47:57.558131 11273 heartbeater.cc:644] Failed to heartbeat to 127.24.215.190:35483 (0 consecutive failures): Network error: Failed to ping master at 127.24.215.190:35483: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
I20260810 14:47:57.558532 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 11130
I20260810 14:47:57.559192 11275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:57.558738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:38763)
0810 14:47:57.558927 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:57.559020 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260810 14:47:57.559311 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:34537
--local_ip_for_outbound_sockets=127.24.215.131
--tserver_master_addrs=127.24.215.190:35483
--webserver_port=38219
--webserver_interface=127.24.215.131
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:45629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:47:57.560187 11275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:57.559738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:42395)
0810 14:47:57.559923 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:57.560024 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 14:47:57.560474 11273 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260810 14:47:57.902653 11276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:47:57.902944 11276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:47:57.903144 11276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:47:57.910363 11276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:47:57.910477 11276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:47:57.910681 11276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:47:57.920367 11276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:45629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:34537
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=38219
--tserver_master_addrs=127.24.215.190:35483
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:47:57.921720 11276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:47:57.923071 11276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:47:57.936652 11281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:57.936776 11282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:47:57.936566 11284 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:47:57.937716 11276 server_base.cc:1034] running on GCE node
I20260810 14:47:57.938318 11276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:47:57.939446 11276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:47:57.940711 11276 hybrid_clock.cc:648] HybridClock initialized: now 1786373277940666 us; error 37 us; skew 500 ppm
I20260810 14:47:57.941037 11276 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:47:57 dist-test-slave-vkjp krb5kdc[9615](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373277, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:47:57.944154 11276 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:47:57.945664 11276 webserver.cc:460] Webserver started at http://127.24.215.131:38219/ using document root <none> and password file <none>
I20260810 14:47:57.946352 11276 fs_manager.cc:359] Metadata directory not provided
I20260810 14:47:57.946504 11276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:47:57.952051 11276 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260810 14:47:57.954756 11294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:57.956187 11276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:47:57.956370 11276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "5f800c59113d401dbece99261430a69e"
format_stamp: "Formatted at 2026-08-10 14:47:51 on dist-test-slave-vkjp"
server_key: "13146e610d1be4baf699389e3ee5443e"
server_key_iv: "b16cea9b57178707ab98b0b23f777a55"
server_key_version: "encryptionkey@0"
I20260810 14:47:57.957365 11276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:47:57.990356 11276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:47:57.994753 11276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:47:57.995050 11276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:47:57.996167 11276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:47:57.997570 11276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:47:57.997718 11276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:57.997876 11276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:47:57.997983 11276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:47:58.032910 11276 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:34537
I20260810 14:47:58.033000 11421 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:34537 every 8 connection(s)
I20260810 14:47:58.034387 11276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:47:58.037317 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 11276
I20260810 14:47:58.039951 11425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:58.037689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:57679)
0810 14:47:58.038638 (+ 949us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:58.038929 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":416,"threads_started":1}
W20260810 14:47:58.040625 11422 heartbeater.cc:644] Failed to heartbeat to 127.24.215.190:35483 (0 consecutive failures): Network error: Failed to ping master at 127.24.215.190:35483: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
I20260810 14:47:58.041458 11425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:58.041121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:37163)
0810 14:47:58.041272 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:58.041352 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 14:47:58.042251 11425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:58.041964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:45509)
0810 14:47:58.042084 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:58.042154 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 14:47:58.042493 11422 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 14:47:58.105737 11427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:58.104811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:60455)
0810 14:47:58.105263 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:58.105405 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":216,"threads_started":1}
I20260810 14:47:58.561882 11428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:58.561045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:57809)
0810 14:47:58.561514 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:58.561636 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":229,"threads_started":1}
I20260810 14:47:59.043893 11429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:59.043051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:44531)
0810 14:47:59.043522 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:59.043637 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":214,"threads_started":1}
I20260810 14:47:59.107547 11430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:59.106735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:37621)
0810 14:47:59.107279 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:59.107369 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":246,"threads_started":1}
I20260810 14:47:59.563526 11431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:47:59.562751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:35863)
0810 14:47:59.563270 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0810 14:47:59.563354 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":238,"threads_started":1}
I20260810 14:48:00.045539 11432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:00.044756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:57671)
0810 14:48:00.045256 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:00.045346 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":270,"threads_started":1}
I20260810 14:48:00.109192 11433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:00.108453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:38329)
0810 14:48:00.108931 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:00.109019 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":363,"thread_start_us":245,"threads_started":1}
I20260810 14:48:00.565342 11434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:00.564571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:44791)
0810 14:48:00.565071 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:00.565157 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":256,"threads_started":1}
I20260810 14:48:01.047154 11435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:01.046411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:59281)
0810 14:48:01.046846 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:01.046924 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":237,"threads_started":1}
I20260810 14:48:01.110958 11436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:01.110157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:33675)
0810 14:48:01.110664 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:01.110756 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":237,"threads_started":1}
I20260810 14:48:01.567153 11437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:01.566357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:59703)
0810 14:48:01.566803 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:01.566886 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":223,"threads_started":1}
I20260810 14:48:02.048763 11439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:02.048042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:39745)
0810 14:48:02.048502 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:02.048588 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":233,"threads_started":1}
I20260810 14:48:02.112802 11441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:02.111994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:47573)
0810 14:48:02.112534 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:02.112626 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":268,"threads_started":1}
I20260810 14:48:02.568770 11442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:02.568054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:36325)
0810 14:48:02.568507 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:02.568590 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":233,"threads_started":1}
I20260810 14:48:03.050619 11443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:03.049840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:39549)
0810 14:48:03.050342 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:03.050434 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":399,"thread_start_us":276,"threads_started":1}
I20260810 14:48:03.114486 11444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:03.113747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:47947)
0810 14:48:03.114222 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:03.114317 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":248,"threads_started":1}
I20260810 14:48:03.570735 11445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:03.569808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:36005)
0810 14:48:03.570378 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:03.570493 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":452,"thread_start_us":297,"threads_started":1}
I20260810 14:48:04.052415 11446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:04.051578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:59027)
0810 14:48:04.052152 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:04.052251 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":324,"threads_started":1}
I20260810 14:48:04.116413 11447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:04.115566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:53743)
0810 14:48:04.116075 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:04.116164 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":221,"threads_started":1}
I20260810 14:48:04.572623 11449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:04.571813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:55393)
0810 14:48:04.572294 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:04.572376 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":217,"threads_started":1}
I20260810 14:48:05.054263 11450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:05.053514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:60135)
0810 14:48:05.053972 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:05.054057 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":199,"threads_started":1}
I20260810 14:48:05.118368 11451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:05.117475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:54773)
0810 14:48:05.117988 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:05.118077 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":257,"threads_started":1}
I20260810 14:48:05.574245 11452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:05.573546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:35023)
0810 14:48:05.573997 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:05.574081 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":235,"threads_started":1}
I20260810 14:48:06.055917 11453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:06.055231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:37055)
0810 14:48:06.055669 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:06.055751 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":228,"threads_started":1}
I20260810 14:48:06.120338 11454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:06.119438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:44227)
0810 14:48:06.120044 (+ 606us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:06.120132 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":317,"threads_started":1}
I20260810 14:48:06.576138 11455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:06.575384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:47913)
0810 14:48:06.575856 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:06.575946 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":231,"threads_started":1}
I20260810 14:48:07.057695 11456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:07.056910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:33765)
0810 14:48:07.057415 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:07.057529 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":250,"threads_started":1}
I20260810 14:48:07.122071 11457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:07.121350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:47695)
0810 14:48:07.121813 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:07.121894 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":266,"threads_started":1}
I20260810 14:48:07.577838 11459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:07.577155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:48807)
0810 14:48:07.577590 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:07.577677 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":234,"threads_started":1}
I20260810 14:48:08.059492 11460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:08.058663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:43489)
0810 14:48:08.059244 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:08.059331 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":331,"threads_started":1}
I20260810 14:48:08.123777 11461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:08.123000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:48843)
0810 14:48:08.123496 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:08.123589 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":274,"threads_started":1}
I20260810 14:48:08.579594 11463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:08.578733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:40033)
0810 14:48:08.579300 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:08.579392 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":238,"threads_started":1}
I20260810 14:48:09.061478 11464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:09.060523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:35633)
0810 14:48:09.060952 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:09.061053 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":205,"threads_started":1}
I20260810 14:48:09.125464 11465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:09.124737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:45637)
0810 14:48:09.125210 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:09.125299 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":229,"threads_started":1}
I20260810 14:48:09.581537 11466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:09.580678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:59787)
0810 14:48:09.581228 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:09.581367 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":298,"threads_started":1}
I20260810 14:48:10.063462 11467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:10.062476 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:41789)
0810 14:48:10.063043 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:10.063214 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":413,"thread_start_us":253,"threads_started":1}
I20260810 14:48:10.127506 11468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:10.126503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:49915)
0810 14:48:10.127047 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:10.127231 (+ 184us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":292,"threads_started":1}
I20260810 14:48:10.583521 11469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:10.582610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:33847)
0810 14:48:10.583190 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:10.583323 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":256,"threads_started":1}
I20260810 14:48:11.065171 11471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:11.064391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:32867)
0810 14:48:11.064888 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:11.064984 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":244,"threads_started":1}
I20260810 14:48:11.129109 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:11.128416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:38841)
0810 14:48:11.128837 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:11.128926 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":211,"threads_started":1}
I20260810 14:48:11.585367 11473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:11.584623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:37661)
0810 14:48:11.585067 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:11.585155 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":218,"threads_started":1}
I20260810 14:48:12.066776 11474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:12.066046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:55717)
0810 14:48:12.066511 (+ 465us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:12.066596 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":231,"threads_started":1}
I20260810 14:48:12.131201 11475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:12.130308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.129:54753)
0810 14:48:12.130795 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:12.130912 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":250,"threads_started":1}
I20260810 14:48:12.587347 11476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:12.586424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.130:43265)
0810 14:48:12.586993 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:12.587166 (+ 173us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":279,"threads_started":1}
I20260810 14:48:13.041854 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 10940
I20260810 14:48:13.060977 11477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:13.043020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55896 (local address 127.24.215.129:32857)
0810 14:48:13.043773 (+ 753us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:13.043788 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:13.044058 (+ 270us) server_negotiation.cc:407] Connection header received
0810 14:48:13.045859 (+ 1801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:13.045869 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:13.046289 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:13.046657 (+ 368us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:13.048275 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:13.049543 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:13.050805 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:13.051557 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:13.051761 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:13.054566 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:13.054576 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:13.054591 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:13.054634 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:13.057431 (+ 2797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:13.058144 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:13.058152 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:13.058158 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:13.058337 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:13.058752 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:13.058758 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:13.058761 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:13.059386 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:13.059716 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:13.060187 (+ 471us) server_negotiation.cc:299] Negotiation successful
0810 14:48:13.060629 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":326,"threads_started":1}
I20260810 14:48:13.062541 11121 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:48:13.068650 11479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:13.067851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:35483 (local address 127.24.215.131:39827)
0810 14:48:13.068366 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:13.068465 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:35483: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":392,"thread_start_us":291,"threads_started":1}
I20260810 14:48:13.148808 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 10940
I20260810 14:48:13.174088 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 11130
I20260810 14:48:13.191704 11481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:13.175165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59268 (local address 127.24.215.130:45417)
0810 14:48:13.175725 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:13.175739 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:13.175778 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:48:13.177412 (+ 1634us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:13.177421 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:13.177829 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:13.178116 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:13.179567 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:13.180813 (+ 1246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:13.182097 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:13.182747 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:13.182879 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:13.185589 (+ 2710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:13.185597 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:13.185612 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:13.185649 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:13.188491 (+ 2842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:13.189146 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:13.189152 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:13.189158 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:13.189334 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:13.189693 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:13.189696 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:13.189698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:13.190178 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:13.190442 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:13.190761 (+ 319us) server_negotiation.cc:299] Negotiation successful
0810 14:48:13.191417 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":401,"thread_start_us":283,"threads_started":1}
I20260810 14:48:13.193094 11268 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:48:13.278173 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 11130
I20260810 14:48:13.300944 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 11276
I20260810 14:48:13.318825 11483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:13.302134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51538 (local address 127.24.215.131:34537)
0810 14:48:13.302832 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:13.302846 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:13.302889 (+ 43us) server_negotiation.cc:407] Connection header received
0810 14:48:13.304622 (+ 1733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:13.304631 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:13.305091 (+ 460us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:13.305403 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:13.306782 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:13.308060 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:13.309314 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:13.309935 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:13.310072 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:13.312901 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:13.312909 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:13.312923 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:13.312960 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:13.315808 (+ 2848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:13.316452 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:13.316458 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:13.316465 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:13.316650 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:13.316998 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:13.317001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:13.317003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:13.317510 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:13.317731 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:13.318102 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 14:48:13.318528 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":538,"thread_start_us":387,"threads_started":1}
I20260810 14:48:13.320254 11416 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:48:13.405627 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 11276
2026-08-10T14:48:13Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (25049 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:48:13 dist-test-slave-vkjp krb5kdc[11487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:48:13 dist-test-slave-vkjp krb5kdc[11487](info): set up 2 sockets
Aug 10 14:48:13 dist-test-slave-vkjp krb5kdc[11487](info): commencing operation
krb5kdc: starting...
W20260810 14:48:14.613461 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.163s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:48:14 dist-test-slave-vkjp krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:48:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:48:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:48:14.891124 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:33161
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:35623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:33161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:48:15.235764 11503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:48:15.236083 11503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:48:15.236177 11503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:48:15.243520 11503 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 14:48:15.243623 11503 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 14:48:15.243661 11503 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 14:48:15.243714 11503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:48:15.243768 11503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:48:15.243845 11503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:48:15.243892 11503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:48:15.243948 11503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:48:15.254009 11503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:35623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:33161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:33161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:48:15.255401 11503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:48:15.256722 11503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:48:15.268893 11508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:48:15.268935 11511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:48:15.268992 11509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:48:15.270005 11503 server_base.cc:1034] running on GCE node
I20260810 14:48:15.270754 11503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:48:15.272184 11503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:48:15.273438 11503 hybrid_clock.cc:648] HybridClock initialized: now 1786373295273370 us; error 35 us; skew 500 ppm
I20260810 14:48:15.273824 11503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:48:15 dist-test-slave-vkjp krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373295, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:48:15.284044 11503 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:48:15.285756 11503 webserver.cc:460] Webserver started at http://127.24.215.190:36225/ using document root <none> and password file <none>
I20260810 14:48:15.286593 11503 fs_manager.cc:359] Metadata directory not provided
I20260810 14:48:15.286715 11503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:48:15.287058 11503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:48:15.290167 11503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "447594d189964ab78597f51ccd72df22"
format_stamp: "Formatted at 2026-08-10 14:48:15 on dist-test-slave-vkjp"
server_key: "9dbc6f37b65d8c44c24cf278d22b2f30"
server_key_iv: "d6d6ed141e80b2a1d17be03d771cc78a"
server_key_version: "encryptionkey@0"
I20260810 14:48:15.290935 11503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "447594d189964ab78597f51ccd72df22"
format_stamp: "Formatted at 2026-08-10 14:48:15 on dist-test-slave-vkjp"
server_key: "9dbc6f37b65d8c44c24cf278d22b2f30"
server_key_iv: "d6d6ed141e80b2a1d17be03d771cc78a"
server_key_version: "encryptionkey@0"
I20260810 14:48:15.296356 11503 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 14:48:15.299964 11521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:15.301395 11503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:48:15.301594 11503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "447594d189964ab78597f51ccd72df22"
format_stamp: "Formatted at 2026-08-10 14:48:15 on dist-test-slave-vkjp"
server_key: "9dbc6f37b65d8c44c24cf278d22b2f30"
server_key_iv: "d6d6ed141e80b2a1d17be03d771cc78a"
server_key_version: "encryptionkey@0"
I20260810 14:48:15.301827 11503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:48:15.327028 11503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:48:15.331771 11503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:48:15.332060 11503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:48:15.351716 11503 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:33161
I20260810 14:48:15.351739 11574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:33161 every 8 connection(s)
I20260810 14:48:15.353101 11503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:48:15.356969 11577 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:48:15.357976 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 11503
I20260810 14:48:15.358196 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:48:15.358896 25438 external_mini_cluster.cc:1428] Setting key b796451d9c77a66ee866d852f801051a
I20260810 14:48:15.371301 11577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Bootstrap starting.
Aug 10 14:48:15 dist-test-slave-vkjp krb5kdc[11487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:48:15.375290 11577 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Neither blocks nor log segments found. Creating new log.
I20260810 14:48:15.376343 11577 log.cc:824] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Log is configured to *not* fsync() on all Append() calls
I20260810 14:48:15.379344 11577 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: No bootstrap required, opened a new log
I20260810 14:48:15.383124 11581 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:15.360971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51008 (local address 127.24.215.190:33161)
0810 14:48:15.361887 (+ 916us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:15.361900 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:15.361940 (+ 40us) server_negotiation.cc:407] Connection header received
0810 14:48:15.363594 (+ 1654us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:15.363620 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:15.364000 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:15.364316 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:15.366221 (+ 1905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:15.367606 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:15.369622 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:15.370305 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:15.370452 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:15.375883 (+ 5431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:15.375891 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:15.375905 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:15.375941 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:15.378969 (+ 3028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:15.379755 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:15.379762 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:15.379771 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:15.379943 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:15.380351 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:15.380356 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:15.380364 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:15.380881 (+ 517us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:15.381236 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:15.381696 (+ 460us) server_negotiation.cc:299] Negotiation successful
0810 14:48:15.382114 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":408,"threads_started":1}
I20260810 14:48:15.388183 11577 raft_consensus.cc:348] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } }
I20260810 14:48:15.388633 11577 raft_consensus.cc:374] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:48:15.388728 11577 raft_consensus.cc:729] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 447594d189964ab78597f51ccd72df22, State: Initialized, Role: FOLLOWER
I20260810 14:48:15.389312 11577 consensus_queue.cc:260] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } }
I20260810 14:48:15.389537 11577 raft_consensus.cc:388] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:48:15.389637 11577 raft_consensus.cc:482] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:48:15.389760 11577 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:48:15.391952 11577 raft_consensus.cc:504] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } }
I20260810 14:48:15.392381 11577 leader_election.cc:302] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 447594d189964ab78597f51ccd72df22; no voters:
I20260810 14:48:15.392962 11577 leader_election.cc:288] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:48:15.393100 11583 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:48:15.394354 11583 raft_consensus.cc:686] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 1 LEADER]: Becoming Leader. State: Replica: 447594d189964ab78597f51ccd72df22, State: Running, Role: LEADER
I20260810 14:48:15.394963 11583 consensus_queue.cc:237] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } }
I20260810 14:48:15.395275 11577 sys_catalog.cc:564] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:48:15.398110 11585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 447594d189964ab78597f51ccd72df22. Latest consensus state: current_term: 1 leader_uuid: "447594d189964ab78597f51ccd72df22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } } }
I20260810 14:48:15.398583 11584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "447594d189964ab78597f51ccd72df22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } } }
I20260810 14:48:15.399154 11584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [sys.catalog]: This master's current role is: LEADER
I20260810 14:48:15.399220 11585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [sys.catalog]: This master's current role is: LEADER
I20260810 14:48:15.400727 11592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:48:15.406241 11592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:48:15.416796 11592 catalog_manager.cc:1347] Generated new cluster ID: 244d1fe036a84d9dae6fd7f3c4d41241
I20260810 14:48:15.416918 11592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:48:15.431567 11592 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:48:15.432901 11592 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:48:15.440590 11592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Generated new TSK 0
I20260810 14:48:15.441674 11592 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:48:15.515184 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:33161
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:35623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:48:15.864369 11606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:48:15.864697 11606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:48:15.864852 11606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:48:15.871968 11606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:48:15.872100 11606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:48:15.872278 11606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:48:15.882161 11606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:35623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:33161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:48:15.883586 11606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:48:15.884883 11606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:48:15.898152 11612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:48:15.898180 11613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:48:15.898214 11615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:48:15.899220 11606 server_base.cc:1034] running on GCE node
I20260810 14:48:15.899844 11606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:48:15.900709 11606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:48:15.901893 11606 hybrid_clock.cc:648] HybridClock initialized: now 1786373295901838 us; error 52 us; skew 500 ppm
I20260810 14:48:15.902194 11606 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:48:15 dist-test-slave-vkjp krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373295, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:48:15.905337 11606 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:48:15.906837 11606 webserver.cc:460] Webserver started at http://127.24.215.129:39855/ using document root <none> and password file <none>
I20260810 14:48:15.907567 11606 fs_manager.cc:359] Metadata directory not provided
I20260810 14:48:15.907675 11606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:48:15.907935 11606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:48:15.910753 11606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "5fd5ddb374244f2dbe0308b439d9af6b"
format_stamp: "Formatted at 2026-08-10 14:48:15 on dist-test-slave-vkjp"
server_key: "3108b40e065ee9ca18cb14df0fe0ae73"
server_key_iv: "b8bb86477f6f63a659084e4a16aa06e6"
server_key_version: "encryptionkey@0"
I20260810 14:48:15.911545 11606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "5fd5ddb374244f2dbe0308b439d9af6b"
format_stamp: "Formatted at 2026-08-10 14:48:15 on dist-test-slave-vkjp"
server_key: "3108b40e065ee9ca18cb14df0fe0ae73"
server_key_iv: "b8bb86477f6f63a659084e4a16aa06e6"
server_key_version: "encryptionkey@0"
I20260810 14:48:15.916707 11606 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:48:15.920145 11622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:15.921428 11606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:48:15.921578 11606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "5fd5ddb374244f2dbe0308b439d9af6b"
format_stamp: "Formatted at 2026-08-10 14:48:15 on dist-test-slave-vkjp"
server_key: "3108b40e065ee9ca18cb14df0fe0ae73"
server_key_iv: "b8bb86477f6f63a659084e4a16aa06e6"
server_key_version: "encryptionkey@0"
I20260810 14:48:15.921742 11606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:48:15.943622 11606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:48:15.948414 11606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:48:15.948740 11606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:48:15.949813 11606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:48:15.951354 11606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:48:15.951442 11606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:15.951618 11606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:48:15.951679 11606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:15.990245 11606 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:46201
I20260810 14:48:15.990335 11758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:46201 every 8 connection(s)
I20260810 14:48:15.991773 11606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:48:15.995304 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 11606
I20260810 14:48:15.995545 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:48:15.996492 25438 external_mini_cluster.cc:1428] Setting key 1b229e242c74c3e032e13ef525ca8459
Aug 10 14:48:16 dist-test-slave-vkjp krb5kdc[11487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373295, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:48:16.016927 11762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:15.994876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:48961 (local address 127.24.215.190:33161)
0810 14:48:15.995477 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:15.995484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:15.996654 (+ 1170us) server_negotiation.cc:407] Connection header received
0810 14:48:15.998956 (+ 2302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:15.998960 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:15.999020 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:15.999172 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:16.001402 (+ 2230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:16.002124 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:16.004731 (+ 2607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:16.005228 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:16.005339 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:16.009826 (+ 4487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:16.009834 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:16.009841 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:16.009894 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:16.013591 (+ 3697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:16.014371 (+ 780us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:16.014377 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:16.014381 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:16.014526 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:16.015017 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:16.015022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:16.015025 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:16.015365 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:16.015529 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:16.016279 (+ 750us) server_negotiation.cc:299] Negotiation successful
0810 14:48:16.016536 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":423,"thread_start_us":281,"threads_started":1}
I20260810 14:48:16.017951 11763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:15.995400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:48961)
0810 14:48:15.996407 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:15.996461 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 14:48:15.998668 (+ 2207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:48:15.999444 (+ 776us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:48:15.999456 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:48:16.000317 (+ 861us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:48:16.001204 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:16.001234 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:16.002296 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:16.002304 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:16.004550 (+ 2246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:16.004568 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:16.005507 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:16.005518 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:16.005866 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:16.006712 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:48:16.006748 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:48:16.009563 (+ 2815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:48:16.013743 (+ 4180us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:48:16.013753 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:48:16.013772 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:48:16.014235 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:48:16.014640 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:48:16.014645 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:48:16.014649 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:48:16.014881 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:48:16.015507 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:48:16.015516 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:48:16.015884 (+ 368us) client_negotiation.cc:769] Sending connection context
0810 14:48:16.016166 (+ 282us) client_negotiation.cc:240] Negotiation successful
0810 14:48:16.016740 (+ 574us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"thread_start_us":351,"threads_started":1}
I20260810 14:48:16.019685 11760 heartbeater.cc:347] Connected to a master server at 127.24.215.190:33161
I20260810 14:48:16.020036 11760 heartbeater.cc:464] Registering TS with master...
I20260810 14:48:16.020998 11760 heartbeater.cc:511] Master 127.24.215.190:33161 requested a full tablet report, sending...
I20260810 14:48:16.023597 11539 ts_manager.cc:194] Registered new tserver with Master: 5fd5ddb374244f2dbe0308b439d9af6b (127.24.215.129:46201)
I20260810 14:48:16.026046 11539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:48961
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:48:16.062426 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:33161
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:35623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:48:16.407419 11766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:48:16.407711 11766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:48:16.407828 11766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:48:16.415319 11766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:48:16.415470 11766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:48:16.415657 11766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:48:16.425662 11766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:35623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:33161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:48:16.427161 11766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:48:16.428581 11766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:48:16.442502 11771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:48:16.442631 11775 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:48:16.442656 11773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:48:16.443917 11766 server_base.cc:1034] running on GCE node
I20260810 14:48:16.444581 11766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:48:16.445593 11766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:48:16.446816 11766 hybrid_clock.cc:648] HybridClock initialized: now 1786373296446769 us; error 35 us; skew 500 ppm
I20260810 14:48:16.447211 11766 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:48:16 dist-test-slave-vkjp krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373296, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:48:16.450555 11766 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:48:16.452276 11766 webserver.cc:460] Webserver started at http://127.24.215.130:34093/ using document root <none> and password file <none>
I20260810 14:48:16.453011 11766 fs_manager.cc:359] Metadata directory not provided
I20260810 14:48:16.453156 11766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:48:16.453516 11766 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:48:16.456781 11766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "78ead0c12f3b475dbe557b7e16104bec"
format_stamp: "Formatted at 2026-08-10 14:48:16 on dist-test-slave-vkjp"
server_key: "6cd86e01202f3d6617963dd6631dec24"
server_key_iv: "814b3e8723fa6bb8a366d35cbb08ec9a"
server_key_version: "encryptionkey@0"
I20260810 14:48:16.457612 11766 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "78ead0c12f3b475dbe557b7e16104bec"
format_stamp: "Formatted at 2026-08-10 14:48:16 on dist-test-slave-vkjp"
server_key: "6cd86e01202f3d6617963dd6631dec24"
server_key_iv: "814b3e8723fa6bb8a366d35cbb08ec9a"
server_key_version: "encryptionkey@0"
I20260810 14:48:16.463238 11766 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260810 14:48:16.466686 11784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:16.468034 11766 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:48:16.468209 11766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "78ead0c12f3b475dbe557b7e16104bec"
format_stamp: "Formatted at 2026-08-10 14:48:16 on dist-test-slave-vkjp"
server_key: "6cd86e01202f3d6617963dd6631dec24"
server_key_iv: "814b3e8723fa6bb8a366d35cbb08ec9a"
server_key_version: "encryptionkey@0"
I20260810 14:48:16.468400 11766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:48:16.505796 11766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:48:16.509807 11766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:48:16.510115 11766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:48:16.511148 11766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:48:16.512466 11766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:48:16.512578 11766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:16.512722 11766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:48:16.512773 11766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:16.546420 11766 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:37741
I20260810 14:48:16.546484 11910 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:37741 every 8 connection(s)
I20260810 14:48:16.547785 11766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:48:16.549998 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 11766
I20260810 14:48:16.550190 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:48:16.550670 25438 external_mini_cluster.cc:1428] Setting key 46f2442b0a05174c3dbc17fc4937c60e
Aug 10 14:48:16 dist-test-slave-vkjp krb5kdc[11487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373296, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:48:16.569123 11915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:16.551292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:39863 (local address 127.24.215.190:33161)
0810 14:48:16.551796 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:16.551802 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:16.551850 (+ 48us) server_negotiation.cc:407] Connection header received
0810 14:48:16.553732 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:16.553740 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:16.553832 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:16.553949 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:16.555521 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:16.556711 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:16.558216 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:16.558971 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:16.559149 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:16.562792 (+ 3643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:16.562801 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:16.562810 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:16.562858 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:16.565750 (+ 2892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:16.566688 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:16.566692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:16.566694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:16.566826 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:16.567381 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:16.567385 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:16.567387 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:16.567660 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:16.567774 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:16.568696 (+ 922us) server_negotiation.cc:299] Negotiation successful
0810 14:48:16.568864 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":263,"threads_started":1}
I20260810 14:48:16.570725 11914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:16.550849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:39863)
0810 14:48:16.551704 (+ 855us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:16.551751 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 14:48:16.553457 (+ 1706us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:48:16.554158 (+ 701us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:48:16.554166 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:48:16.554698 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:48:16.555333 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:16.555360 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:16.556857 (+ 1497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:16.556860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:16.558065 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:16.558078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:16.559231 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:16.559236 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:16.559457 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:16.560163 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:48:16.560185 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:48:16.562555 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:48:16.565893 (+ 3338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:48:16.565905 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:48:16.565930 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:48:16.566570 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:48:16.566937 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:48:16.566945 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:48:16.566951 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:48:16.567263 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:48:16.567791 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:48:16.567801 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:48:16.568280 (+ 479us) client_negotiation.cc:769] Sending connection context
0810 14:48:16.568611 (+ 331us) client_negotiation.cc:240] Negotiation successful
0810 14:48:16.569201 (+ 590us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":374,"threads_started":1}
I20260810 14:48:16.572340 11911 heartbeater.cc:347] Connected to a master server at 127.24.215.190:33161
I20260810 14:48:16.572667 11911 heartbeater.cc:464] Registering TS with master...
I20260810 14:48:16.573500 11911 heartbeater.cc:511] Master 127.24.215.190:33161 requested a full tablet report, sending...
I20260810 14:48:16.575023 11539 ts_manager.cc:194] Registered new tserver with Master: 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741)
I20260810 14:48:16.575893 11539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:39863
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:48:16.614244 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:33161
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:35623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:48:16.942369 11918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:48:16.942683 11918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:48:16.942847 11918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:48:16.950076 11918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:48:16.950228 11918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:48:16.950421 11918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:48:16.960141 11918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:35623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:33161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:48:16.961498 11918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:48:16.962819 11918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:48:16.975652 11923 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:48:16.975703 11926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:48:16.975723 11924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:48:16.977140 11918 server_base.cc:1034] running on GCE node
I20260810 14:48:16.977784 11918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:48:16.978703 11918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:48:16.979924 11918 hybrid_clock.cc:648] HybridClock initialized: now 1786373296979878 us; error 53 us; skew 500 ppm
I20260810 14:48:16.980235 11918 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:48:16 dist-test-slave-vkjp krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373296, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:48:16.983354 11918 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:48:16.984808 11918 webserver.cc:460] Webserver started at http://127.24.215.131:39063/ using document root <none> and password file <none>
I20260810 14:48:16.985491 11918 fs_manager.cc:359] Metadata directory not provided
I20260810 14:48:16.985630 11918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:48:16.985935 11918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:48:16.988843 11918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "0ec912c539e74652906603d968389e37"
format_stamp: "Formatted at 2026-08-10 14:48:16 on dist-test-slave-vkjp"
server_key: "28dca4ddeca3a8004b1e0b2440b1cb5e"
server_key_iv: "b3398cd12aa833b3bf51200d2c744969"
server_key_version: "encryptionkey@0"
I20260810 14:48:16.989663 11918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "0ec912c539e74652906603d968389e37"
format_stamp: "Formatted at 2026-08-10 14:48:16 on dist-test-slave-vkjp"
server_key: "28dca4ddeca3a8004b1e0b2440b1cb5e"
server_key_iv: "b3398cd12aa833b3bf51200d2c744969"
server_key_version: "encryptionkey@0"
I20260810 14:48:16.994822 11918 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 14:48:16.998420 11933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:16.999747 11918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:48:16.999905 11918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "0ec912c539e74652906603d968389e37"
format_stamp: "Formatted at 2026-08-10 14:48:16 on dist-test-slave-vkjp"
server_key: "28dca4ddeca3a8004b1e0b2440b1cb5e"
server_key_iv: "b3398cd12aa833b3bf51200d2c744969"
server_key_version: "encryptionkey@0"
I20260810 14:48:17.000106 11918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:48:17.020502 11918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:48:17.024859 11918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:48:17.025218 11918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:48:17.026177 11918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:48:17.027590 11918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:48:17.027688 11918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:17.027828 11918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:48:17.027896 11918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:17.029806 11760 heartbeater.cc:503] Master 127.24.215.190:33161 was elected leader, sending a full tablet report...
I20260810 14:48:17.061733 11918 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:39053
I20260810 14:48:17.061775 12061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:39053 every 8 connection(s)
I20260810 14:48:17.063040 11918 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:48:17.070135 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 11918
I20260810 14:48:17.070362 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:48:17.070968 25438 external_mini_cluster.cc:1428] Setting key 02f68ef7c689822a6134210e6a9be174
Aug 10 14:48:17 dist-test-slave-vkjp krb5kdc[11487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373296, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:48:17.083695 11915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.065789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:40351 (local address 127.24.215.190:33161)
0810 14:48:17.065987 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:17.065991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:17.067097 (+ 1106us) server_negotiation.cc:407] Connection header received
0810 14:48:17.068957 (+ 1860us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:17.068964 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:17.069059 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:17.069165 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:17.071115 (+ 1950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.072153 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.074282 (+ 2129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.074677 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.074773 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.078353 (+ 3580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:17.078360 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:17.078365 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:17.078400 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:17.081110 (+ 2710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:17.081759 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:17.081763 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:17.081766 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:17.081897 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:17.082248 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:17.082252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:17.082253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:17.082506 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:17.082630 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:17.083287 (+ 657us) server_negotiation.cc:299] Negotiation successful
0810 14:48:17.083469 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 14:48:17.084762 12064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.066195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:40351)
0810 14:48:17.066902 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:17.066946 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:48:17.068685 (+ 1739us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:48:17.069401 (+ 716us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:48:17.069409 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:48:17.070042 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:48:17.070852 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:17.070877 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.072363 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.072371 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:17.074119 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:17.074137 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.074879 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.074885 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:17.075071 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.075745 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:48:17.075772 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:48:17.078166 (+ 2394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:48:17.081223 (+ 3057us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:48:17.081229 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:48:17.081241 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:48:17.081663 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:48:17.081986 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:48:17.081989 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:48:17.081992 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:48:17.082161 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:48:17.082607 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:48:17.082613 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:48:17.082914 (+ 301us) client_negotiation.cc:769] Sending connection context
0810 14:48:17.083193 (+ 279us) client_negotiation.cc:240] Negotiation successful
0810 14:48:17.083575 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":292,"threads_started":1}
I20260810 14:48:17.086302 12062 heartbeater.cc:347] Connected to a master server at 127.24.215.190:33161
I20260810 14:48:17.086651 12062 heartbeater.cc:464] Registering TS with master...
I20260810 14:48:17.087482 12062 heartbeater.cc:511] Master 127.24.215.190:33161 requested a full tablet report, sending...
I20260810 14:48:17.088999 11539 ts_manager.cc:194] Registered new tserver with Master: 0ec912c539e74652906603d968389e37 (127.24.215.131:39053)
I20260810 14:48:17.089825 11539 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:40351
I20260810 14:48:17.098943 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:48:17.116856 11915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.103621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51014 (local address 127.24.215.190:33161)
0810 14:48:17.103814 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:17.103820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:17.104290 (+ 470us) server_negotiation.cc:407] Connection header received
0810 14:48:17.104506 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:17.104511 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:17.104574 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:17.104673 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:17.106308 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.107146 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.108581 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.109038 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.109176 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.111710 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:17.111716 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:17.111719 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:17.111752 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:17.114156 (+ 2404us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:17.114815 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:17.114822 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:17.114826 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:17.114975 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:17.115376 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:17.115383 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:17.115387 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:17.115723 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:17.115991 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:17.116287 (+ 296us) server_negotiation.cc:299] Negotiation successful
0810 14:48:17.116500 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 14:48:17.120818 11539 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:48:17.128521 11539 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:48:17.165036 12077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.140742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46201 (local address 127.0.0.1:53722)
0810 14:48:17.155123 (+ 14381us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:17.155172 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 14:48:17.155394 (+ 222us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:48:17.156683 (+ 1289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:48:17.156692 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:48:17.156736 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:48:17.157237 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:17.157255 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.159148 (+ 1893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.159154 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:17.161623 (+ 2469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:17.161635 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.163572 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.163579 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:17.164128 (+ 549us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.164198 (+ 70us) client_negotiation.cc:769] Sending connection context
0810 14:48:17.164324 (+ 126us) client_negotiation.cc:240] Negotiation successful
0810 14:48:17.164504 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14071,"mutex_wait_us":23,"thread_start_us":14714,"threads_started":1}
I20260810 14:48:17.165064 12075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.141109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49146 (local address 127.24.215.130:37741)
0810 14:48:17.154203 (+ 13094us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:17.154218 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:17.155431 (+ 1213us) server_negotiation.cc:407] Connection header received
0810 14:48:17.156943 (+ 1512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:17.156956 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:17.157226 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:17.157446 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:48:17.158476 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.159844 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.162658 (+ 2814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.163956 (+ 1298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.164141 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.164327 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:17.164586 (+ 259us) server_negotiation.cc:299] Negotiation successful
0810 14:48:17.164751 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12820,"thread_start_us":13122,"threads_started":1}
I20260810 14:48:17.165124 12074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.141272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:37741 (local address 127.0.0.1:49146)
0810 14:48:17.155248 (+ 13976us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:17.155273 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:48:17.156247 (+ 974us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:48:17.157754 (+ 1507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:48:17.157762 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:48:17.157797 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:48:17.158279 (+ 482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:17.158295 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.160033 (+ 1738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.160038 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:17.162489 (+ 2451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:17.162502 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.164191 (+ 1689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.164197 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:17.164391 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.164416 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 14:48:17.164525 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 14:48:17.164675 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13767,"mutex_wait_us":35,"thread_start_us":13383,"threads_started":1}
I20260810 14:48:17.166347 12076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.141123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53722 (local address 127.24.215.129:46201)
0810 14:48:17.155974 (+ 14851us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:17.155982 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:17.156022 (+ 40us) server_negotiation.cc:407] Connection header received
0810 14:48:17.156149 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:17.156158 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:17.156428 (+ 270us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:17.157474 (+ 1046us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:48:17.157708 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.158978 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.161779 (+ 2801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.163140 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.165608 (+ 2468us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.165771 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:17.165861 (+ 90us) server_negotiation.cc:299] Negotiation successful
0810 14:48:17.166024 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14652,"thread_start_us":14481,"threads_started":1}
I20260810 14:48:17.168546 12078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.155886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:39053 (local address 127.0.0.1:51736)
0810 14:48:17.156461 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:17.156481 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:48:17.156616 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:48:17.159490 (+ 2874us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:48:17.159495 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:48:17.159526 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:48:17.160341 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:17.160359 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.162385 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.162390 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:17.164710 (+ 2320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:17.164725 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.167643 (+ 2918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.167650 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:17.167888 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.167927 (+ 39us) client_negotiation.cc:769] Sending connection context
0810 14:48:17.168037 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 14:48:17.168165 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":453,"mutex_wait_us":66,"thread_start_us":477,"threads_started":1}
I20260810 14:48:17.168588 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.156310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51736 (local address 127.24.215.131:39053)
0810 14:48:17.158922 (+ 2612us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:17.158927 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:17.158952 (+ 25us) server_negotiation.cc:407] Connection header received
0810 14:48:17.159022 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:17.159026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:17.159284 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:17.159481 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:48:17.160577 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.162222 (+ 1645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.164913 (+ 2691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.167444 (+ 2531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.167656 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.167879 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:17.168083 (+ 204us) server_negotiation.cc:299] Negotiation successful
0810 14:48:17.168238 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2468,"thread_start_us":2284,"threads_started":1}
I20260810 14:48:17.169291 11839 tablet_service.cc:1467] Processing CreateTablet for tablet 254bcc2b53d643aa9910648979234d84 (DEFAULT_TABLE table=test-table [id=3597a0fefb624d7abbbfa17e1419acbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:48:17.169852 11682 tablet_service.cc:1467] Processing CreateTablet for tablet 254bcc2b53d643aa9910648979234d84 (DEFAULT_TABLE table=test-table [id=3597a0fefb624d7abbbfa17e1419acbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:48:17.171124 11839 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 254bcc2b53d643aa9910648979234d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:48:17.171376 11682 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 254bcc2b53d643aa9910648979234d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:48:17.172206 11987 tablet_service.cc:1467] Processing CreateTablet for tablet 254bcc2b53d643aa9910648979234d84 (DEFAULT_TABLE table=test-table [id=3597a0fefb624d7abbbfa17e1419acbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:48:17.173413 11987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 254bcc2b53d643aa9910648979234d84. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:48:17.184696 12080 tablet_bootstrap.cc:492] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37: Bootstrap starting.
I20260810 14:48:17.187914 12082 tablet_bootstrap.cc:492] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec: Bootstrap starting.
I20260810 14:48:17.189519 12080 tablet_bootstrap.cc:654] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37: Neither blocks nor log segments found. Creating new log.
I20260810 14:48:17.190871 12080 log.cc:824] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37: Log is configured to *not* fsync() on all Append() calls
I20260810 14:48:17.191668 12082 tablet_bootstrap.cc:654] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec: Neither blocks nor log segments found. Creating new log.
I20260810 14:48:17.192394 12081 tablet_bootstrap.cc:492] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b: Bootstrap starting.
I20260810 14:48:17.193522 12082 log.cc:824] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec: Log is configured to *not* fsync() on all Append() calls
I20260810 14:48:17.194099 12080 tablet_bootstrap.cc:492] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37: No bootstrap required, opened a new log
I20260810 14:48:17.194386 12080 ts_tablet_manager.cc:1397] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 14:48:17.196504 12081 tablet_bootstrap.cc:654] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b: Neither blocks nor log segments found. Creating new log.
I20260810 14:48:17.198148 12081 log.cc:824] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b: Log is configured to *not* fsync() on all Append() calls
I20260810 14:48:17.201572 12081 tablet_bootstrap.cc:492] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b: No bootstrap required, opened a new log
I20260810 14:48:17.201802 12082 tablet_bootstrap.cc:492] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec: No bootstrap required, opened a new log
I20260810 14:48:17.201840 12081 ts_tablet_manager.cc:1397] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 14:48:17.202092 12082 ts_tablet_manager.cc:1397] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260810 14:48:17.204310 12080 raft_consensus.cc:348] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } }
I20260810 14:48:17.204793 12080 raft_consensus.cc:374] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:48:17.204901 12080 raft_consensus.cc:729] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec912c539e74652906603d968389e37, State: Initialized, Role: FOLLOWER
I20260810 14:48:17.205572 12080 consensus_queue.cc:260] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } }
I20260810 14:48:17.206750 12062 heartbeater.cc:503] Master 127.24.215.190:33161 was elected leader, sending a full tablet report...
I20260810 14:48:17.207408 12080 ts_tablet_manager.cc:1428] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
I20260810 14:48:17.211503 12082 raft_consensus.cc:348] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } }
I20260810 14:48:17.211704 12081 raft_consensus.cc:348] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } }
I20260810 14:48:17.211994 12082 raft_consensus.cc:374] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:48:17.212119 12081 raft_consensus.cc:374] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:48:17.212119 12082 raft_consensus.cc:729] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78ead0c12f3b475dbe557b7e16104bec, State: Initialized, Role: FOLLOWER
I20260810 14:48:17.212235 12081 raft_consensus.cc:729] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd5ddb374244f2dbe0308b439d9af6b, State: Initialized, Role: FOLLOWER
I20260810 14:48:17.212764 12082 consensus_queue.cc:260] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } }
I20260810 14:48:17.212949 12081 consensus_queue.cc:260] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } }
I20260810 14:48:17.213901 11911 heartbeater.cc:503] Master 127.24.215.190:33161 was elected leader, sending a full tablet report...
I20260810 14:48:17.214640 12082 ts_tablet_manager.cc:1428] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260810 14:48:17.219473 12081 ts_tablet_manager.cc:1428] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b: Time spent starting tablet: real 0.017s user 0.012s sys 0.005s
W20260810 14:48:17.244649 11761 tablet.cc:2369] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:48:17.299965 11913 tablet.cc:2369] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:48:17.314774 12063 tablet.cc:2369] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:48:17.501828 12088 raft_consensus.cc:482] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:48:17.502195 12088 raft_consensus.cc:504] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } }
I20260810 14:48:17.504287 12088 leader_election.cc:288] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741), 0ec912c539e74652906603d968389e37 (127.24.215.131:39053)
I20260810 14:48:17.511684 12075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.504664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:58969 (local address 127.24.215.130:37741)
0810 14:48:17.504944 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:17.504954 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:17.505809 (+ 855us) server_negotiation.cc:407] Connection header received
0810 14:48:17.506111 (+ 302us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:17.506118 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:17.506199 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:17.506285 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:48:17.507040 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.507819 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.509840 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.510663 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.510829 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.510922 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:17.511275 (+ 353us) server_negotiation.cc:299] Negotiation successful
0810 14:48:17.511406 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 14:48:17.511713 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.504986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:48903 (local address 127.24.215.131:39053)
0810 14:48:17.505179 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:17.505187 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:17.505962 (+ 775us) server_negotiation.cc:407] Connection header received
0810 14:48:17.506146 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:17.506150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:17.506208 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:17.506307 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:48:17.507180 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.508027 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.509876 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.510674 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.510863 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.510941 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:17.511335 (+ 394us) server_negotiation.cc:299] Negotiation successful
0810 14:48:17.511463 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260810 14:48:17.511854 12089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.504542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:37741 (local address 127.24.215.129:58969)
0810 14:48:17.505691 (+ 1149us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:17.505709 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:48:17.505907 (+ 198us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:48:17.506313 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:48:17.506318 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:48:17.506342 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:48:17.506875 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:17.506889 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.507953 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.507958 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:17.509682 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:17.509695 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.510900 (+ 1205us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.510907 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:17.511065 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.511137 (+ 72us) client_negotiation.cc:769] Sending connection context
0810 14:48:17.511221 (+ 84us) client_negotiation.cc:240] Negotiation successful
0810 14:48:17.511366 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":924,"mutex_wait_us":39,"thread_start_us":649,"threads_started":1}
I20260810 14:48:17.512112 12090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.504846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:39053 (local address 127.24.215.129:48903)
0810 14:48:17.505854 (+ 1008us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:17.505880 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0810 14:48:17.506020 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:48:17.506341 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:48:17.506346 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:48:17.506458 (+ 112us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:48:17.506918 (+ 460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:17.506929 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.508178 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.508182 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:17.509691 (+ 1509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:17.509704 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.510941 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.510949 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:17.511136 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.511169 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:48:17.511272 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 14:48:17.511401 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":875,"mutex_wait_us":26,"thread_start_us":623,"threads_started":1}
I20260810 14:48:17.513000 11861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "254bcc2b53d643aa9910648979234d84" candidate_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78ead0c12f3b475dbe557b7e16104bec" is_pre_election: true
I20260810 14:48:17.513181 12010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "254bcc2b53d643aa9910648979234d84" candidate_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec912c539e74652906603d968389e37" is_pre_election: true
I20260810 14:48:17.513631 11861 raft_consensus.cc:2454] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fd5ddb374244f2dbe0308b439d9af6b in term 0.
I20260810 14:48:17.513648 12010 raft_consensus.cc:2454] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5fd5ddb374244f2dbe0308b439d9af6b in term 0.
I20260810 14:48:17.514278 11625 leader_election.cc:302] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd5ddb374244f2dbe0308b439d9af6b, 78ead0c12f3b475dbe557b7e16104bec; no voters:
I20260810 14:48:17.514681 12088 raft_consensus.cc:2781] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:48:17.514849 12088 raft_consensus.cc:482] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:48:17.514969 12088 raft_consensus.cc:3037] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:48:17.517210 12088 raft_consensus.cc:504] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } }
I20260810 14:48:17.517858 12088 leader_election.cc:288] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 1 election: Requested vote from peers 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741), 0ec912c539e74652906603d968389e37 (127.24.215.131:39053)
I20260810 14:48:17.518205 11861 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "254bcc2b53d643aa9910648979234d84" candidate_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78ead0c12f3b475dbe557b7e16104bec"
I20260810 14:48:17.518445 11861 raft_consensus.cc:3037] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:48:17.518415 12010 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "254bcc2b53d643aa9910648979234d84" candidate_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ec912c539e74652906603d968389e37"
I20260810 14:48:17.518620 12010 raft_consensus.cc:3037] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:48:17.521859 11861 raft_consensus.cc:2454] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd5ddb374244f2dbe0308b439d9af6b in term 1.
I20260810 14:48:17.521857 12010 raft_consensus.cc:2454] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5fd5ddb374244f2dbe0308b439d9af6b in term 1.
I20260810 14:48:17.522259 11625 leader_election.cc:302] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5fd5ddb374244f2dbe0308b439d9af6b, 78ead0c12f3b475dbe557b7e16104bec; no voters:
I20260810 14:48:17.522565 12088 raft_consensus.cc:2781] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:48:17.523138 12088 raft_consensus.cc:686] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 LEADER]: Becoming Leader. State: Replica: 5fd5ddb374244f2dbe0308b439d9af6b, State: Running, Role: LEADER
I20260810 14:48:17.523625 12088 consensus_queue.cc:237] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } }
I20260810 14:48:17.528249 11537 catalog_manager.cc:5495] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b reported cstate change: term changed from 0 to 1, leader changed from <none> to 5fd5ddb374244f2dbe0308b439d9af6b (127.24.215.129). New cstate: current_term: 1 leader_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } health_report { overall_health: UNKNOWN } } }
I20260810 14:48:17.624329 12076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.618408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53724 (local address 127.24.215.129:46201)
0810 14:48:17.618709 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:17.618718 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:17.619066 (+ 348us) server_negotiation.cc:407] Connection header received
0810 14:48:17.619263 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:17.619268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:17.619377 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:17.619510 (+ 133us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:48:17.619932 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.620683 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.622104 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.622523 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.622618 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.623162 (+ 544us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:48:17.623718 (+ 556us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:48:17.623836 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:17.623934 (+ 98us) server_negotiation.cc:299] Negotiation successful
0810 14:48:17.624000 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 14:48:17.632721 12010 raft_consensus.cc:1261] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37 [term 1 FOLLOWER]: Refusing update from remote peer 5fd5ddb374244f2dbe0308b439d9af6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:48:17.632722 11861 raft_consensus.cc:1261] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec [term 1 FOLLOWER]: Refusing update from remote peer 5fd5ddb374244f2dbe0308b439d9af6b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:48:17.633713 12088 consensus_queue.cc:1035] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:48:17.634008 12091 consensus_queue.cc:1035] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [LEADER]: Connected to new peer: Peer: permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:48:17.648545 12098 mvcc.cc:205] Tried to move back new op lower bound from 7316985027093491712 to 7316985026660839424. Current Snapshot: MvccSnapshot[applied={T|T < 7316985027093491712}]
I20260810 14:48:17.668445 11537 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:48:17.668725 11537 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:48:17.672425 11537 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Sending DeleteTablet for 3 replicas of tablet 254bcc2b53d643aa9910648979234d84
I20260810 14:48:17.673777 11839 tablet_service.cc:1514] Processing DeleteTablet for tablet 254bcc2b53d643aa9910648979234d84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:48:17 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:49146
I20260810 14:48:17.673868 11987 tablet_service.cc:1514] Processing DeleteTablet for tablet 254bcc2b53d643aa9910648979234d84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:48:17 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:51736
I20260810 14:48:17.674083 11682 tablet_service.cc:1514] Processing DeleteTablet for tablet 254bcc2b53d643aa9910648979234d84 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:48:17 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:53722
I20260810 14:48:17.675159 12109 tablet_replica.cc:330] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec: stopping tablet replica
I20260810 14:48:17.675500 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 11606
I20260810 14:48:17.675657 12109 raft_consensus.cc:2229] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:48:17.676222 12109 raft_consensus.cc:2258] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:48:17.677975 12111 tablet_replica.cc:330] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b: stopping tablet replica
I20260810 14:48:17.678568 12111 raft_consensus.cc:2229] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:48:17.679438 12111 raft_consensus.cc:2258] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:48:17.684615 12111 ts_tablet_manager.cc:1905] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:48:17.687675 12110 tablet_replica.cc:330] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37: stopping tablet replica
I20260810 14:48:17.688139 12110 raft_consensus.cc:2229] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:48:17.688604 12109 ts_tablet_manager.cc:1905] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:48:17.691519 12110 raft_consensus.cc:2258] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:48:17.693531 12110 ts_tablet_manager.cc:1905] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:48:17.694692 12109 ts_tablet_manager.cc:1918] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:48:17.694895 12109 log.cc:1196] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/254bcc2b53d643aa9910648979234d84
I20260810 14:48:17.696059 12109 ts_tablet_manager.cc:1939] T 254bcc2b53d643aa9910648979234d84 P 78ead0c12f3b475dbe557b7e16104bec: Deleting consensus metadata
I20260810 14:48:17.697742 12111 ts_tablet_manager.cc:1918] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:48:17.697983 12111 log.cc:1196] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/254bcc2b53d643aa9910648979234d84
I20260810 14:48:17.698063 11523 catalog_manager.cc:4841] TS 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741): tablet 254bcc2b53d643aa9910648979234d84 (table test-table [id=3597a0fefb624d7abbbfa17e1419acbb]) successfully deleted
I20260810 14:48:17.698617 12111 ts_tablet_manager.cc:1939] T 254bcc2b53d643aa9910648979234d84 P 5fd5ddb374244f2dbe0308b439d9af6b: Deleting consensus metadata
I20260810 14:48:17.700590 11525 catalog_manager.cc:4841] TS 5fd5ddb374244f2dbe0308b439d9af6b (127.24.215.129:46201): tablet 254bcc2b53d643aa9910648979234d84 (table test-table [id=3597a0fefb624d7abbbfa17e1419acbb]) successfully deleted
I20260810 14:48:17.700784 12110 ts_tablet_manager.cc:1918] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:48:17.700973 12110 log.cc:1196] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/254bcc2b53d643aa9910648979234d84
I20260810 14:48:17.701623 12110 ts_tablet_manager.cc:1939] T 254bcc2b53d643aa9910648979234d84 P 0ec912c539e74652906603d968389e37: Deleting consensus metadata
I20260810 14:48:17.703408 11525 catalog_manager.cc:4841] TS 0ec912c539e74652906603d968389e37 (127.24.215.131:39053): tablet 254bcc2b53d643aa9910648979234d84 (table test-table [id=3597a0fefb624d7abbbfa17e1419acbb]) successfully deleted
Aug 10 14:48:17 dist-test-slave-vkjp krb5kdc[11487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:48:17.713320 12076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.676868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53728 (local address 127.24.215.129:46201)
0810 14:48:17.677059 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:17.677066 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:17.687881 (+ 10815us) server_negotiation.cc:407] Connection header received
0810 14:48:17.688107 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:17.688110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:17.688177 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:17.691743 (+ 3566us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:17.691928 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.693206 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.700805 (+ 7599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.701595 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.701707 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.707115 (+ 5408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:17.707133 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:17.707155 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:17.707203 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:17.710319 (+ 3116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:17.711198 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:17.711205 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:17.711208 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:17.711374 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:17.711779 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:17.711782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:17.711784 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:17.712182 (+ 398us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:17.712418 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:17.712816 (+ 398us) server_negotiation.cc:299] Negotiation successful
0810 14:48:17.712997 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:48:17.714119 11753 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:48:17.813284 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 11606
I20260810 14:48:17.837385 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 11766
Aug 10 14:48:17 dist-test-slave-vkjp krb5kdc[11487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:48:17.853417 12075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.838221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49162 (local address 127.24.215.130:37741)
0810 14:48:17.838417 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:17.838422 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:17.838442 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:48:17.838576 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:17.838580 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:17.838652 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:17.838768 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:17.840304 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.841087 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.842586 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.842997 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.843190 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.847731 (+ 4541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:17.847747 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:17.847764 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:17.847800 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:17.850719 (+ 2919us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:17.851519 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:17.851526 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:17.851529 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:17.851656 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:17.851990 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:17.851993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:17.851995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:17.852379 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:17.852604 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:17.852864 (+ 260us) server_negotiation.cc:299] Negotiation successful
0810 14:48:17.853046 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 14:48:17.854197 11906 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:48:17.943365 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 11766
I20260810 14:48:17.964912 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 11918
Aug 10 14:48:17 dist-test-slave-vkjp krb5kdc[11487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373294, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:48:17.980679 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:17.965744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51742 (local address 127.24.215.131:39053)
0810 14:48:17.965946 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:17.965951 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:17.965970 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:48:17.966153 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:17.966157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:17.966225 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:17.966311 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:17.967892 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.968598 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.969909 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:17.970290 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:17.970388 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:17.975261 (+ 4873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:17.975279 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:17.975296 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:17.975339 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:17.977972 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:17.978622 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:17.978629 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:17.978631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:17.978776 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:17.979160 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:17.979168 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:17.979173 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:17.979554 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:17.979862 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:17.980127 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 14:48:17.980291 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 14:48:17.981324 12057 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:48:18.070079 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 11918
I20260810 14:48:18.089797 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 11503
I20260810 14:48:18.090952 11570 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:48:18.206831 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 11503
I20260810 14:48:38.233542 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:33161
--webserver_interface=127.24.215.190
--webserver_port=36225
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:35623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:33161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:48:38.579471 12117 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:48:38.579775 12117 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:48:38.579869 12117 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:48:38.587528 12117 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 14:48:38.587639 12117 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 14:48:38.587678 12117 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 14:48:38.587723 12117 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:48:38.587779 12117 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:48:38.587847 12117 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:48:38.587893 12117 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:48:38.587954 12117 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:48:38.598073 12117 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:35623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:33161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:33161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=36225
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:48:38.599416 12117 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:48:38.600775 12117 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:48:38.627569 12126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:48:38.627493 12124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:48:38.627614 12123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:48:38.628572 12117 server_base.cc:1034] running on GCE node
I20260810 14:48:38.629555 12117 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:48:38.631295 12117 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:48:38.632606 12117 hybrid_clock.cc:648] HybridClock initialized: now 1786373318632531 us; error 71 us; skew 500 ppm
I20260810 14:48:38.633042 12117 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:48:38 dist-test-slave-vkjp krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373318, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:48:38.636747 12117 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:48:38.638434 12117 webserver.cc:460] Webserver started at http://127.24.215.190:36225/ using document root <none> and password file <none>
I20260810 14:48:38.639294 12117 fs_manager.cc:359] Metadata directory not provided
I20260810 14:48:38.639410 12117 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:48:38.645670 12117 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260810 14:48:38.648993 12135 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:38.650487 12117 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260810 14:48:38.650694 12117 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "447594d189964ab78597f51ccd72df22"
format_stamp: "Formatted at 2026-08-10 14:48:15 on dist-test-slave-vkjp"
server_key: "9dbc6f37b65d8c44c24cf278d22b2f30"
server_key_iv: "d6d6ed141e80b2a1d17be03d771cc78a"
server_key_version: "encryptionkey@0"
I20260810 14:48:38.651970 12117 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:48:38.667685 12117 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:48:38.672109 12117 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:48:38.672421 12117 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:48:38.692706 12117 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:33161
I20260810 14:48:38.692726 12193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:33161 every 8 connection(s)
I20260810 14:48:38.694391 12117 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:48:38.701802 12194 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:48:38.703060 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 12117
I20260810 14:48:38.704135 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:46201
--local_ip_for_outbound_sockets=127.24.215.129
--tserver_master_addrs=127.24.215.190:33161
--webserver_port=39855
--webserver_interface=127.24.215.129
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:35623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:48:38.706995 12194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Bootstrap starting.
I20260810 14:48:38.729761 12194 log.cc:824] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Log is configured to *not* fsync() on all Append() calls
I20260810 14:48:38.742661 12194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:48:38.743319 12194 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Bootstrap complete.
I20260810 14:48:38.753062 12194 raft_consensus.cc:348] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } }
I20260810 14:48:38.753906 12194 raft_consensus.cc:729] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 447594d189964ab78597f51ccd72df22, State: Initialized, Role: FOLLOWER
I20260810 14:48:38.754588 12194 consensus_queue.cc:260] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } }
I20260810 14:48:38.754839 12194 raft_consensus.cc:388] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:48:38.754998 12194 raft_consensus.cc:482] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:48:38.755230 12194 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 1 FOLLOWER]: Advancing to term 2
I20260810 14:48:38.760036 12194 raft_consensus.cc:504] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } }
I20260810 14:48:38.760589 12194 leader_election.cc:302] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 447594d189964ab78597f51ccd72df22; no voters:
I20260810 14:48:38.761276 12194 leader_election.cc:288] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260810 14:48:38.761509 12198 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 2 FOLLOWER]: Leader election won for term 2
I20260810 14:48:38.762730 12198 raft_consensus.cc:686] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 2 LEADER]: Becoming Leader. State: Replica: 447594d189964ab78597f51ccd72df22, State: Running, Role: LEADER
I20260810 14:48:38.763540 12194 sys_catalog.cc:564] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:48:38.763608 12198 consensus_queue.cc:237] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } }
I20260810 14:48:38.766322 12200 sys_catalog.cc:455] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 447594d189964ab78597f51ccd72df22. Latest consensus state: current_term: 2 leader_uuid: "447594d189964ab78597f51ccd72df22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } } }
I20260810 14:48:38.766716 12200 sys_catalog.cc:458] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [sys.catalog]: This master's current role is: LEADER
I20260810 14:48:38.767277 12199 sys_catalog.cc:455] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "447594d189964ab78597f51ccd72df22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } } }
I20260810 14:48:38.767583 12199 sys_catalog.cc:458] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [sys.catalog]: This master's current role is: LEADER
I20260810 14:48:38.772274 12205 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:48:38.781467 12205 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:48:38.783031 12205 catalog_manager.cc:1259] Loaded cluster ID: 244d1fe036a84d9dae6fd7f3c4d41241
I20260810 14:48:38.783499 12205 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:48:38.787575 12205 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:48:38.789279 12205 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Loaded TSK: 0
I20260810 14:48:38.833947 12205 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Generated new TSK 1
I20260810 14:48:38.834501 12205 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 14:48:39.097147 12196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:48:39.097491 12196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:48:39.097666 12196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:48:39.105353 12196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:48:39.105525 12196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:48:39.105698 12196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:48:39.115875 12196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:35623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:46201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=39855
--tserver_master_addrs=127.24.215.190:33161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:48:39.117256 12196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:48:39.118606 12196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:48:39.134186 12224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:48:39.134399 12225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:48:39.135401 12196 server_base.cc:1034] running on GCE node
W20260810 14:48:39.135713 12227 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:48:39.136513 12196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:48:39.137692 12196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:48:39.138935 12196 hybrid_clock.cc:648] HybridClock initialized: now 1786373319138893 us; error 35 us; skew 500 ppm
I20260810 14:48:39.139321 12196 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:48:39 dist-test-slave-vkjp krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373319, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:48:39.143343 12196 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:48:39.145128 12196 webserver.cc:460] Webserver started at http://127.24.215.129:39855/ using document root <none> and password file <none>
I20260810 14:48:39.145845 12196 fs_manager.cc:359] Metadata directory not provided
I20260810 14:48:39.145996 12196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:48:39.151945 12196 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:48:39.154946 12234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:39.156558 12196 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:48:39.156775 12196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "5fd5ddb374244f2dbe0308b439d9af6b"
format_stamp: "Formatted at 2026-08-10 14:48:15 on dist-test-slave-vkjp"
server_key: "3108b40e065ee9ca18cb14df0fe0ae73"
server_key_iv: "b8bb86477f6f63a659084e4a16aa06e6"
server_key_version: "encryptionkey@0"
I20260810 14:48:39.157889 12196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:48:39.169631 12196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:48:39.174212 12196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:48:39.174511 12196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:48:39.175624 12196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:48:39.177156 12196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:48:39.177285 12196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:39.177417 12196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:48:39.177507 12196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:39.212006 12196 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:46201
I20260810 14:48:39.212098 12365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:46201 every 8 connection(s)
I20260810 14:48:39.213513 12196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:48:39.216764 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 12196
I20260810 14:48:39.217603 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:37741
--local_ip_for_outbound_sockets=127.24.215.130
--tserver_master_addrs=127.24.215.190:33161
--webserver_port=34093
--webserver_interface=127.24.215.130
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:35623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:48:39 dist-test-slave-vkjp krb5kdc[11487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373319, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:48:39.242545 12369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:39.216605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:56287)
0810 14:48:39.217841 (+ 1236us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:39.217901 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0810 14:48:39.220735 (+ 2834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:48:39.223077 (+ 2342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:48:39.223089 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:48:39.223743 (+ 654us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:48:39.224285 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:39.224302 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:39.226800 (+ 2498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:39.226804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:39.228104 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:39.228118 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:39.229458 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:39.229467 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:39.229796 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:39.230657 (+ 861us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:48:39.230683 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:48:39.233257 (+ 2574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:48:39.238320 (+ 5063us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:48:39.238328 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:48:39.238341 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:48:39.238784 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:48:39.239378 (+ 594us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:48:39.239381 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:48:39.239384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:48:39.239567 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:48:39.240562 (+ 995us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:48:39.240567 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:48:39.240857 (+ 290us) client_negotiation.cc:769] Sending connection context
0810 14:48:39.241120 (+ 263us) client_negotiation.cc:240] Negotiation successful
0810 14:48:39.241529 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":914,"thread_start_us":720,"threads_started":1}
I20260810 14:48:39.243237 12368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:39.216586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:56287 (local address 127.24.215.190:33161)
0810 14:48:39.217378 (+ 792us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:39.217399 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:39.218044 (+ 645us) server_negotiation.cc:407] Connection header received
0810 14:48:39.221536 (+ 3492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:39.221558 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:39.222041 (+ 483us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:39.222493 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:39.224561 (+ 2068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:39.226622 (+ 2061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:39.228263 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:39.229187 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:39.229339 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:39.233608 (+ 4269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:39.233622 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:39.233647 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:39.233709 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:39.238022 (+ 4313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:39.238960 (+ 938us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:39.238973 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:39.238984 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:39.239257 (+ 273us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:39.239702 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:39.239710 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:39.239716 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:39.240424 (+ 708us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:39.240633 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:39.241417 (+ 784us) server_negotiation.cc:299] Negotiation successful
0810 14:48:39.241897 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":496,"thread_start_us":315,"threads_started":1}
I20260810 14:48:39.245954 12366 heartbeater.cc:347] Connected to a master server at 127.24.215.190:33161
I20260810 14:48:39.246271 12366 heartbeater.cc:464] Registering TS with master...
I20260810 14:48:39.246989 12366 heartbeater.cc:511] Master 127.24.215.190:33161 requested a full tablet report, sending...
I20260810 14:48:39.249230 12156 ts_manager.cc:194] Registered new tserver with Master: 5fd5ddb374244f2dbe0308b439d9af6b (127.24.215.129:46201)
I20260810 14:48:39.250734 12156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:56287
W20260810 14:48:39.580255 12370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:48:39.580596 12370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:48:39.580807 12370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:48:39.589610 12370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:48:39.589769 12370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:48:39.590008 12370 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:48:39.601099 12370 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:35623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:37741
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=34093
--tserver_master_addrs=127.24.215.190:33161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:48:39.602391 12370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:48:39.603719 12370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:48:39.617161 12376 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:48:39.617308 12380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:48:39.617259 12377 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:48:39.618197 12370 server_base.cc:1034] running on GCE node
I20260810 14:48:39.618799 12370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:48:39.620110 12370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:48:39.621340 12370 hybrid_clock.cc:648] HybridClock initialized: now 1786373319621289 us; error 42 us; skew 500 ppm
I20260810 14:48:39.621701 12370 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:48:39 dist-test-slave-vkjp krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373319, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:48:39.624886 12370 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:48:39.626394 12370 webserver.cc:460] Webserver started at http://127.24.215.130:34093/ using document root <none> and password file <none>
I20260810 14:48:39.627138 12370 fs_manager.cc:359] Metadata directory not provided
I20260810 14:48:39.627303 12370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:48:39.632612 12370 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.003s
I20260810 14:48:39.635403 12388 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:39.636726 12370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:48:39.636921 12370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "78ead0c12f3b475dbe557b7e16104bec"
format_stamp: "Formatted at 2026-08-10 14:48:16 on dist-test-slave-vkjp"
server_key: "6cd86e01202f3d6617963dd6631dec24"
server_key_iv: "814b3e8723fa6bb8a366d35cbb08ec9a"
server_key_version: "encryptionkey@0"
I20260810 14:48:39.637889 12370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:48:39.658826 12370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:48:39.663000 12370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:48:39.663306 12370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:48:39.664330 12370 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:48:39.665695 12370 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:48:39.665819 12370 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:39.665947 12370 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:48:39.666033 12370 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:39.700362 12370 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:37741
I20260810 14:48:39.700417 12513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:37741 every 8 connection(s)
I20260810 14:48:39.701953 12370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:48:39.703258 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 12370
I20260810 14:48:39.703950 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:39053
--local_ip_for_outbound_sockets=127.24.215.131
--tserver_master_addrs=127.24.215.190:33161
--webserver_port=39063
--webserver_interface=127.24.215.131
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:35623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 10 14:48:39 dist-test-slave-vkjp krb5kdc[11487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373319, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:48:39.727236 12368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:39.706614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:35469 (local address 127.24.215.190:33161)
0810 14:48:39.706858 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:39.706867 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:39.708524 (+ 1657us) server_negotiation.cc:407] Connection header received
0810 14:48:39.711375 (+ 2851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:39.711381 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:39.711493 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:39.711606 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:39.713230 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:39.713962 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:39.715651 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:39.716130 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:39.716270 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:39.720185 (+ 3915us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:39.720196 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:39.720205 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:39.720253 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:39.723885 (+ 3632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:39.724596 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:39.724604 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:39.724611 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:39.724775 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:39.725210 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:39.725218 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:39.725224 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:39.725634 (+ 410us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:39.725790 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:39.726489 (+ 699us) server_negotiation.cc:299] Negotiation successful
0810 14:48:39.726764 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 14:48:39.728046 12518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:39.705925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:35469)
0810 14:48:39.708352 (+ 2427us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:39.708401 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0810 14:48:39.710968 (+ 2567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:48:39.711893 (+ 925us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:48:39.711907 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:48:39.712455 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:48:39.713047 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:39.713066 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:39.714154 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:39.714160 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:39.715504 (+ 1344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:39.715517 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:39.716383 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:39.716391 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:39.716624 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:39.717351 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:48:39.717386 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:48:39.719901 (+ 2515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:48:39.724039 (+ 4138us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:48:39.724047 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:48:39.724060 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:48:39.724455 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:48:39.724895 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:48:39.724898 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:48:39.724900 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:48:39.725078 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:48:39.725755 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:48:39.725761 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:48:39.726085 (+ 324us) client_negotiation.cc:769] Sending connection context
0810 14:48:39.726310 (+ 225us) client_negotiation.cc:240] Negotiation successful
0810 14:48:39.726718 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2136,"thread_start_us":1880,"threads_started":1}
I20260810 14:48:39.729561 12515 heartbeater.cc:347] Connected to a master server at 127.24.215.190:33161
I20260810 14:48:39.729926 12515 heartbeater.cc:464] Registering TS with master...
I20260810 14:48:39.730903 12515 heartbeater.cc:511] Master 127.24.215.190:33161 requested a full tablet report, sending...
I20260810 14:48:39.732721 12156 ts_manager.cc:194] Registered new tserver with Master: 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741)
I20260810 14:48:39.734043 12156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:35469
W20260810 14:48:40.047497 12519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:48:40.047760 12519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:48:40.047851 12519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:48:40.054883 12519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:48:40.054994 12519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:48:40.055181 12519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:48:40.065223 12519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:35623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:39053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=39063
--tserver_master_addrs=127.24.215.190:33161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:48:40.066502 12519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:48:40.067883 12519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:48:40.081233 12526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:48:40.081293 12528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:48:40.081321 12524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:48:40.082170 12519 server_base.cc:1034] running on GCE node
I20260810 14:48:40.082891 12519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:48:40.084154 12519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:48:40.085368 12519 hybrid_clock.cc:648] HybridClock initialized: now 1786373320085303 us; error 57 us; skew 500 ppm
I20260810 14:48:40.085803 12519 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:48:40 dist-test-slave-vkjp krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373320, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:48:40.090176 12519 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:48:40.091988 12519 webserver.cc:460] Webserver started at http://127.24.215.131:39063/ using document root <none> and password file <none>
I20260810 14:48:40.092953 12519 fs_manager.cc:359] Metadata directory not provided
I20260810 14:48:40.093075 12519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:48:40.098245 12519 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 14:48:40.100935 12536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:40.102334 12519 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 14:48:40.102516 12519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "0ec912c539e74652906603d968389e37"
format_stamp: "Formatted at 2026-08-10 14:48:16 on dist-test-slave-vkjp"
server_key: "28dca4ddeca3a8004b1e0b2440b1cb5e"
server_key_iv: "b3398cd12aa833b3bf51200d2c744969"
server_key_version: "encryptionkey@0"
I20260810 14:48:40.103545 12519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:48:40.125093 12519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:48:40.129458 12519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:48:40.129731 12519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:48:40.130697 12519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:48:40.132172 12519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:48:40.132315 12519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:40.132448 12519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:48:40.132536 12519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:48:40.166086 12519 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:39053
I20260810 14:48:40.166173 12667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:39053 every 8 connection(s)
I20260810 14:48:40.167685 12519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:48:40.177584 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 12519
Aug 10 14:48:40 dist-test-slave-vkjp krb5kdc[11487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373320, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:48:40.189231 12368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.170233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:47515 (local address 127.24.215.190:33161)
0810 14:48:40.170413 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:40.170418 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:40.171578 (+ 1160us) server_negotiation.cc:407] Connection header received
0810 14:48:40.173484 (+ 1906us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:40.173490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:40.173610 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:40.173723 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:40.175340 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.176089 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.177732 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.178255 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.178421 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.182348 (+ 3927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:40.182354 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:40.182359 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:40.182399 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:40.185676 (+ 3277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:40.186538 (+ 862us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:40.186542 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:40.186544 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:40.186658 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:40.187186 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:40.187190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:40.187192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:40.187448 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:40.187570 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:40.188592 (+ 1022us) server_negotiation.cc:299] Negotiation successful
0810 14:48:40.188827 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:48:40.192467 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.170629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:47515)
0810 14:48:40.171415 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:40.171460 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0810 14:48:40.173208 (+ 1748us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:48:40.174001 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:48:40.174009 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:48:40.174528 (+ 519us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:48:40.175163 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:40.175186 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.176258 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.176264 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:40.177571 (+ 1307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:40.177584 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.178537 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.178548 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:40.178877 (+ 329us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.179746 (+ 869us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:48:40.179775 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:48:40.182146 (+ 2371us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:48:40.185797 (+ 3651us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:48:40.185806 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:48:40.185824 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:48:40.186422 (+ 598us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:48:40.186781 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:48:40.186785 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:48:40.186789 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:48:40.187050 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:48:40.187589 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:48:40.187599 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:48:40.188103 (+ 504us) client_negotiation.cc:769] Sending connection context
0810 14:48:40.189050 (+ 947us) client_negotiation.cc:240] Negotiation successful
0810 14:48:40.190587 (+ 1537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":284,"threads_started":1}
I20260810 14:48:40.194274 12668 heartbeater.cc:347] Connected to a master server at 127.24.215.190:33161
I20260810 14:48:40.194624 12668 heartbeater.cc:464] Registering TS with master...
I20260810 14:48:40.195562 12668 heartbeater.cc:511] Master 127.24.215.190:33161 requested a full tablet report, sending...
I20260810 14:48:40.197089 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.178649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38452 (local address 127.24.215.190:33161)
0810 14:48:40.179574 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:40.179580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:40.179723 (+ 143us) server_negotiation.cc:407] Connection header received
0810 14:48:40.179959 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:40.179965 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:40.180038 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:40.180147 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:40.182099 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.183311 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.184827 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.185577 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.185707 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.189186 (+ 3479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:40.189195 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:40.189201 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:40.189246 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:40.192877 (+ 3631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:40.193833 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:40.193839 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:40.193842 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:40.194001 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:40.194485 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:40.194492 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:40.194496 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:40.194925 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:40.196418 (+ 1493us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:40.196572 (+ 154us) server_negotiation.cc:299] Negotiation successful
0810 14:48:40.196754 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":800,"thread_start_us":638,"threads_started":1}
I20260810 14:48:40.197455 12156 ts_manager.cc:194] Registered new tserver with Master: 0ec912c539e74652906603d968389e37 (127.24.215.131:39053)
I20260810 14:48:40.198503 12156 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:47515
I20260810 14:48:40.200152 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:48:40.209489 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.201596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38464 (local address 127.24.215.190:33161)
0810 14:48:40.201876 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:40.201915 (+ 39us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:40.202721 (+ 806us) server_negotiation.cc:407] Connection header received
0810 14:48:40.202960 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:40.202968 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:40.203196 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:40.203370 (+ 174us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:48:40.203810 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.204960 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.206515 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.207318 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.207467 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.208361 (+ 894us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:48:40.208810 (+ 449us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0810 14:48:40.209189 (+ 379us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:38464: token expired
Metrics: {"server-negotiator.queue_time_us":43}
W20260810 14:48:40.209625 12672 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:38464: token expired
W20260810 14:48:40.210300 12065 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.24.215.190:33161)
I20260810 14:48:40.228950 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.210897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38468 (local address 127.24.215.190:33161)
0810 14:48:40.211114 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:40.211122 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:40.211227 (+ 105us) server_negotiation.cc:407] Connection header received
0810 14:48:40.211476 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:40.211481 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:40.211575 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:40.211667 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:40.213673 (+ 2006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.214909 (+ 1236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.216991 (+ 2082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.217771 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.217920 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.222040 (+ 4120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:40.222047 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:40.222051 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:40.222090 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:40.225885 (+ 3795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:40.226879 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:40.226883 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:40.226885 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:40.227009 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:40.227511 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:40.227515 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:40.227517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:40.227832 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:40.228032 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:40.228504 (+ 472us) server_negotiation.cc:299] Negotiation successful
0810 14:48:40.228671 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:48:40.233424 12155 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38468:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:48:40.242645 12155 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:48:40.253873 12366 heartbeater.cc:503] Master 127.24.215.190:33161 was elected leader, sending a full tablet report...
I20260810 14:48:40.268424 12679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.254656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46201 (local address 127.0.0.1:57968)
0810 14:48:40.255693 (+ 1037us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:40.255747 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 14:48:40.255932 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:48:40.257171 (+ 1239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:48:40.257182 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:48:40.257224 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:48:40.257786 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:40.257804 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.260448 (+ 2644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.260457 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:40.263502 (+ 3045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:40.263581 (+ 79us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.267031 (+ 3450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.267041 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:40.267377 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.267459 (+ 82us) client_negotiation.cc:769] Sending connection context
0810 14:48:40.267600 (+ 141us) client_negotiation.cc:240] Negotiation successful
0810 14:48:40.267868 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":761,"thread_start_us":709,"threads_started":1}
I20260810 14:48:40.270143 12680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.255361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57968 (local address 127.24.215.129:46201)
0810 14:48:40.256505 (+ 1144us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:40.256514 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:40.256543 (+ 29us) server_negotiation.cc:407] Connection header received
0810 14:48:40.256640 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:40.256649 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:40.256905 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:40.258020 (+ 1115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:48:40.258256 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.260238 (+ 1982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.263829 (+ 3591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.266694 (+ 2865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.269067 (+ 2373us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.269340 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:40.269433 (+ 93us) server_negotiation.cc:299] Negotiation successful
0810 14:48:40.269669 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":960,"thread_start_us":1564,"threads_started":1}
I20260810 14:48:40.275401 12292 tablet_service.cc:1467] Processing CreateTablet for tablet 5f46f335050845d5b4e51c33e66e1d23 (DEFAULT_TABLE table=test-table [id=d71bc8edd9ed45faa19dd3f633b836d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:48:40.276860 12292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f46f335050845d5b4e51c33e66e1d23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:48:40.279621 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.256656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54422 (local address 127.24.215.131:39053)
0810 14:48:40.268566 (+ 11910us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:40.268579 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:40.270481 (+ 1902us) server_negotiation.cc:407] Connection header received
0810 14:48:40.270595 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:40.270604 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:40.270882 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:40.271621 (+ 739us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:48:40.271907 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.273693 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.276296 (+ 2603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.277797 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.278014 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.278205 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:40.279054 (+ 849us) server_negotiation.cc:299] Negotiation successful
0810 14:48:40.279256 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11666,"thread_start_us":11922,"threads_started":1}
I20260810 14:48:40.279747 12681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.255881 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:37741 (local address 127.0.0.1:40470)
0810 14:48:40.263019 (+ 7138us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:40.263044 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:48:40.263933 (+ 889us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:48:40.271194 (+ 7261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:48:40.271197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:48:40.271218 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:48:40.271727 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:40.271739 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.273922 (+ 2183us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.273926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:40.276109 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:40.276123 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.278067 (+ 1944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.278074 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:40.278289 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.278313 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 14:48:40.278409 (+ 96us) client_negotiation.cc:240] Negotiation successful
0810 14:48:40.278550 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6936,"mutex_wait_us":73,"thread_start_us":6753,"threads_started":1}
I20260810 14:48:40.279822 12683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.256490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40470 (local address 127.24.215.130:37741)
0810 14:48:40.270623 (+ 14133us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:40.270631 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:40.270655 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:48:40.270734 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:40.270740 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:40.270964 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:40.271355 (+ 391us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:48:40.271924 (+ 569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.273769 (+ 1845us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.276312 (+ 2543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.277797 (+ 1485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.279042 (+ 1245us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.279232 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:40.279301 (+ 69us) server_negotiation.cc:299] Negotiation successful
0810 14:48:40.279476 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13955,"thread_start_us":14566,"threads_started":1}
I20260810 14:48:40.280346 12679 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.256161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:39053 (local address 127.0.0.1:54422)
0810 14:48:40.268792 (+ 12631us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:40.268812 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:48:40.270316 (+ 1504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:48:40.271147 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:48:40.271155 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:48:40.271182 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:48:40.271435 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:40.271448 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.273886 (+ 2438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.273895 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:40.276050 (+ 2155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:40.276071 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.278000 (+ 1929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.278006 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:40.278221 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.278257 (+ 36us) client_negotiation.cc:769] Sending connection context
0810 14:48:40.278378 (+ 121us) client_negotiation.cc:240] Negotiation successful
0810 14:48:40.278540 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12541,"mutex_wait_us":7,"thread_start_us":14703,"threads_started":1}
I20260810 14:48:40.283718 12591 tablet_service.cc:1467] Processing CreateTablet for tablet 5f46f335050845d5b4e51c33e66e1d23 (DEFAULT_TABLE table=test-table [id=d71bc8edd9ed45faa19dd3f633b836d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:48:40.283927 12444 tablet_service.cc:1467] Processing CreateTablet for tablet 5f46f335050845d5b4e51c33e66e1d23 (DEFAULT_TABLE table=test-table [id=d71bc8edd9ed45faa19dd3f633b836d4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:48:40.285164 12591 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f46f335050845d5b4e51c33e66e1d23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:48:40.285260 12444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f46f335050845d5b4e51c33e66e1d23. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:48:40.293855 12686 tablet_bootstrap.cc:492] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b: Bootstrap starting.
I20260810 14:48:40.297749 12686 tablet_bootstrap.cc:654] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b: Neither blocks nor log segments found. Creating new log.
I20260810 14:48:40.298518 12688 tablet_bootstrap.cc:492] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37: Bootstrap starting.
I20260810 14:48:40.299666 12686 log.cc:824] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b: Log is configured to *not* fsync() on all Append() calls
I20260810 14:48:40.299804 12689 tablet_bootstrap.cc:492] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec: Bootstrap starting.
I20260810 14:48:40.302558 12688 tablet_bootstrap.cc:654] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37: Neither blocks nor log segments found. Creating new log.
I20260810 14:48:40.303920 12689 tablet_bootstrap.cc:654] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec: Neither blocks nor log segments found. Creating new log.
I20260810 14:48:40.304019 12688 log.cc:824] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37: Log is configured to *not* fsync() on all Append() calls
I20260810 14:48:40.305163 12689 log.cc:824] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec: Log is configured to *not* fsync() on all Append() calls
I20260810 14:48:40.306061 12686 tablet_bootstrap.cc:492] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b: No bootstrap required, opened a new log
I20260810 14:48:40.306370 12686 ts_tablet_manager.cc:1397] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b: Time spent bootstrapping tablet: real 0.013s user 0.006s sys 0.003s
I20260810 14:48:40.306841 12688 tablet_bootstrap.cc:492] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37: No bootstrap required, opened a new log
I20260810 14:48:40.307204 12688 ts_tablet_manager.cc:1397] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260810 14:48:40.308408 12689 tablet_bootstrap.cc:492] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec: No bootstrap required, opened a new log
I20260810 14:48:40.308660 12689 ts_tablet_manager.cc:1397] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec: Time spent bootstrapping tablet: real 0.009s user 0.008s sys 0.000s
I20260810 14:48:40.314921 12688 raft_consensus.cc:348] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } }
I20260810 14:48:40.315387 12688 raft_consensus.cc:374] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:48:40.315536 12688 raft_consensus.cc:729] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ec912c539e74652906603d968389e37, State: Initialized, Role: FOLLOWER
I20260810 14:48:40.316187 12688 consensus_queue.cc:260] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } }
I20260810 14:48:40.317378 12668 heartbeater.cc:503] Master 127.24.215.190:33161 was elected leader, sending a full tablet report...
I20260810 14:48:40.317543 12686 raft_consensus.cc:348] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } }
I20260810 14:48:40.317953 12688 ts_tablet_manager.cc:1428] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37: Time spent starting tablet: real 0.011s user 0.007s sys 0.003s
I20260810 14:48:40.317991 12686 raft_consensus.cc:374] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:48:40.318141 12686 raft_consensus.cc:729] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd5ddb374244f2dbe0308b439d9af6b, State: Initialized, Role: FOLLOWER
I20260810 14:48:40.318789 12686 consensus_queue.cc:260] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } }
I20260810 14:48:40.320577 12686 ts_tablet_manager.cc:1428] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260810 14:48:40.320387 12689 raft_consensus.cc:348] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } }
I20260810 14:48:40.320813 12689 raft_consensus.cc:374] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:48:40.320919 12689 raft_consensus.cc:729] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78ead0c12f3b475dbe557b7e16104bec, State: Initialized, Role: FOLLOWER
I20260810 14:48:40.321559 12689 consensus_queue.cc:260] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } }
I20260810 14:48:40.322834 12515 heartbeater.cc:503] Master 127.24.215.190:33161 was elected leader, sending a full tablet report...
I20260810 14:48:40.323464 12689 ts_tablet_manager.cc:1428] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
W20260810 14:48:40.419387 12669 tablet.cc:2369] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:48:40.460454 12516 tablet.cc:2369] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:48:40.465909 12367 tablet.cc:2369] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:48:40.482959 12693 raft_consensus.cc:482] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:48:40.483366 12693 raft_consensus.cc:504] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } }
I20260810 14:48:40.485215 12693 leader_election.cc:288] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5fd5ddb374244f2dbe0308b439d9af6b (127.24.215.129:46201), 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741)
I20260810 14:48:40.491360 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.485561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46201 (local address 127.24.215.131:59099)
0810 14:48:40.485805 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:40.485832 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0810 14:48:40.486001 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:48:40.486508 (+ 507us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:48:40.486514 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:48:40.486551 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:48:40.486766 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:40.486774 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.488057 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.488063 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:40.489406 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:40.489421 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.490544 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.490551 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:40.490742 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.490767 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 14:48:40.490875 (+ 108us) client_negotiation.cc:240] Negotiation successful
0810 14:48:40.491032 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":80}
I20260810 14:48:40.491364 12680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.485903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:59099 (local address 127.24.215.129:46201)
0810 14:48:40.486140 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:40.486146 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:40.486171 (+ 25us) server_negotiation.cc:407] Connection header received
0810 14:48:40.486257 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:40.486261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:40.486331 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:40.486473 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:48:40.486968 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.487890 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.489588 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.490376 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.490566 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.490649 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:40.490918 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 14:48:40.491126 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260810 14:48:40.492304 12683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.485964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:40989 (local address 127.24.215.130:37741)
0810 14:48:40.486168 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:40.486173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:40.486946 (+ 773us) server_negotiation.cc:407] Connection header received
0810 14:48:40.487224 (+ 278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:40.487229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:40.487302 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:40.487414 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:48:40.488007 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.488749 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.490424 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.491337 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.491508 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.491580 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:40.491915 (+ 335us) server_negotiation.cc:299] Negotiation successful
0810 14:48:40.492047 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 14:48:40.492384 12696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.485750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:37741 (local address 127.24.215.131:40989)
0810 14:48:40.486807 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:40.486827 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:48:40.487007 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:48:40.487428 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:48:40.487439 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:48:40.487463 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:48:40.487844 (+ 381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:40.487853 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.488897 (+ 1044us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.488903 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:40.490255 (+ 1352us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:48:40.490268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.491555 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.491562 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:48:40.491745 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.491775 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 14:48:40.491887 (+ 112us) client_negotiation.cc:240] Negotiation successful
0810 14:48:40.492011 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"thread_start_us":645,"threads_started":1}
I20260810 14:48:40.492635 12315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f46f335050845d5b4e51c33e66e1d23" candidate_uuid: "0ec912c539e74652906603d968389e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" is_pre_election: true
I20260810 14:48:40.493305 12315 raft_consensus.cc:2454] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec912c539e74652906603d968389e37 in term 0.
I20260810 14:48:40.493392 12466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f46f335050845d5b4e51c33e66e1d23" candidate_uuid: "0ec912c539e74652906603d968389e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78ead0c12f3b475dbe557b7e16104bec" is_pre_election: true
I20260810 14:48:40.493906 12466 raft_consensus.cc:2454] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ec912c539e74652906603d968389e37 in term 0.
I20260810 14:48:40.494213 12542 leader_election.cc:302] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec912c539e74652906603d968389e37, 5fd5ddb374244f2dbe0308b439d9af6b; no voters:
I20260810 14:48:40.494629 12693 raft_consensus.cc:2781] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:48:40.494807 12693 raft_consensus.cc:482] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:48:40.494921 12693 raft_consensus.cc:3037] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:48:40.497246 12693 raft_consensus.cc:504] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } }
I20260810 14:48:40.498160 12693 leader_election.cc:288] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [CANDIDATE]: Term 1 election: Requested vote from peers 5fd5ddb374244f2dbe0308b439d9af6b (127.24.215.129:46201), 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741)
I20260810 14:48:40.498713 12315 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f46f335050845d5b4e51c33e66e1d23" candidate_uuid: "0ec912c539e74652906603d968389e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5fd5ddb374244f2dbe0308b439d9af6b"
I20260810 14:48:40.498713 12466 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f46f335050845d5b4e51c33e66e1d23" candidate_uuid: "0ec912c539e74652906603d968389e37" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78ead0c12f3b475dbe557b7e16104bec"
I20260810 14:48:40.498971 12315 raft_consensus.cc:3037] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:48:40.498965 12466 raft_consensus.cc:3037] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:48:40.502442 12315 raft_consensus.cc:2454] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec912c539e74652906603d968389e37 in term 1.
I20260810 14:48:40.502449 12466 raft_consensus.cc:2454] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ec912c539e74652906603d968389e37 in term 1.
I20260810 14:48:40.502955 12539 leader_election.cc:302] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ec912c539e74652906603d968389e37, 78ead0c12f3b475dbe557b7e16104bec; no voters:
I20260810 14:48:40.503391 12693 raft_consensus.cc:2781] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:48:40.503916 12693 raft_consensus.cc:686] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [term 1 LEADER]: Becoming Leader. State: Replica: 0ec912c539e74652906603d968389e37, State: Running, Role: LEADER
I20260810 14:48:40.504415 12693 consensus_queue.cc:237] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } }
I20260810 14:48:40.509692 12155 catalog_manager.cc:5495] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ec912c539e74652906603d968389e37 (127.24.215.131). New cstate: current_term: 1 leader_uuid: "0ec912c539e74652906603d968389e37" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } health_report { overall_health: UNKNOWN } } }
I20260810 14:48:40.570040 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.564356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60066 (local address 127.24.215.131:39053)
0810 14:48:40.564558 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:40.564565 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:40.564610 (+ 45us) server_negotiation.cc:407] Connection header received
0810 14:48:40.564871 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:40.564875 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:40.564958 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:40.565116 (+ 158us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:48:40.565515 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.566245 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.567809 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.568223 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.568367 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.568913 (+ 546us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:48:40.569415 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:48:40.569601 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:40.569682 (+ 81us) server_negotiation.cc:299] Negotiation successful
0810 14:48:40.569766 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260810 14:48:40.578359 12466 raft_consensus.cc:1261] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec [term 1 FOLLOWER]: Refusing update from remote peer 0ec912c539e74652906603d968389e37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:48:40.578369 12315 raft_consensus.cc:1261] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Refusing update from remote peer 0ec912c539e74652906603d968389e37: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:48:40.579502 12693 consensus_queue.cc:1035] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:48:40.579821 12697 consensus_queue.cc:1035] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:48:40.591650 12702 mvcc.cc:205] Tried to move back new op lower bound from 7316985121078575104 to 7316985120790159360. Current Snapshot: MvccSnapshot[applied={T|T < 7316985121078575104}]
I20260810 14:48:40.612056 12155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:48:40.612310 12155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38468:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:48:40.616223 12155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Sending DeleteTablet for 3 replicas of tablet 5f46f335050845d5b4e51c33e66e1d23
I20260810 14:48:40.617635 12444 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f46f335050845d5b4e51c33e66e1d23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:48:40 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:40470
I20260810 14:48:40.617712 12292 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f46f335050845d5b4e51c33e66e1d23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:48:40 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:57968
I20260810 14:48:40.617830 12591 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f46f335050845d5b4e51c33e66e1d23 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:48:40 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:54422
I20260810 14:48:40.618700 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 12196
I20260810 14:48:40.653829 12680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.620176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52228 (local address 127.24.215.129:46201)
0810 14:48:40.620415 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:40.620423 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:40.620444 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:48:40.620515 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:40.620524 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:40.620627 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:40.620768 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:40.623633 (+ 2865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.625170 (+ 1537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.628953 (+ 3783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.629914 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.630052 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.641188 (+ 11136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:40.641215 (+ 27us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:40.641244 (+ 29us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:40.641309 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:40.646879 (+ 5570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:40.648333 (+ 1454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:40.648348 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:40.648354 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:40.648596 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:40.649631 (+ 1035us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:40.649640 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:40.649646 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:40.650367 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:40.650763 (+ 396us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:40.652860 (+ 2097us) server_negotiation.cc:299] Negotiation successful
0810 14:48:40.653199 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 14:48:40.654914 12361 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:48:40.657622 12717 tablet_replica.cc:330] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37: stopping tablet replica
I20260810 14:48:40.658288 12717 raft_consensus.cc:2229] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:48:40.659571 12717 raft_consensus.cc:2258] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:48:40.660490 12714 tablet_replica.cc:330] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec: stopping tablet replica
I20260810 14:48:40.663226 12717 ts_tablet_manager.cc:1905] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:48:40.660907 12714 raft_consensus.cc:2229] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:48:40.665117 12714 raft_consensus.cc:2258] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:48:40.673698 12714 ts_tablet_manager.cc:1905] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:48:40.675384 12717 ts_tablet_manager.cc:1918] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:48:40.675580 12717 log.cc:1196] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/5f46f335050845d5b4e51c33e66e1d23
I20260810 14:48:40.676165 12717 ts_tablet_manager.cc:1939] T 5f46f335050845d5b4e51c33e66e1d23 P 0ec912c539e74652906603d968389e37: Deleting consensus metadata
I20260810 14:48:40.678114 12139 catalog_manager.cc:4841] TS 0ec912c539e74652906603d968389e37 (127.24.215.131:39053): tablet 5f46f335050845d5b4e51c33e66e1d23 (table test-table [id=d71bc8edd9ed45faa19dd3f633b836d4]) successfully deleted
I20260810 14:48:40.681277 12714 ts_tablet_manager.cc:1918] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:48:40.681443 12714 log.cc:1196] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/5f46f335050845d5b4e51c33e66e1d23
I20260810 14:48:40.682049 12714 ts_tablet_manager.cc:1939] T 5f46f335050845d5b4e51c33e66e1d23 P 78ead0c12f3b475dbe557b7e16104bec: Deleting consensus metadata
I20260810 14:48:40.684019 12137 catalog_manager.cc:4841] TS 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741): tablet 5f46f335050845d5b4e51c33e66e1d23 (table test-table [id=d71bc8edd9ed45faa19dd3f633b836d4]) successfully deleted
I20260810 14:48:40.779649 12716 tablet_replica.cc:330] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b: stopping tablet replica
I20260810 14:48:40.779982 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 12196
I20260810 14:48:40.801844 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 12370
W20260810 14:48:40.802012 12139 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.215.129:46201 (error 108)
I20260810 14:48:40.804551 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.803824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46201 (local address 127.0.0.1:52240)
0810 14:48:40.804288 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:40.804391 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":261,"threads_started":1}
W20260810 14:48:40.804885 12139 catalog_manager.cc:4584] TS 5fd5ddb374244f2dbe0308b439d9af6b (127.24.215.129:46201): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5f46f335050845d5b4e51c33e66e1d23: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46201: connect: Connection refused (error 111)
I20260810 14:48:40.819185 12683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.802727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34486 (local address 127.24.215.130:37741)
0810 14:48:40.802980 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:40.802988 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:40.803015 (+ 27us) server_negotiation.cc:407] Connection header received
0810 14:48:40.803262 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:40.803269 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:40.803360 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:40.803468 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:40.805523 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.806316 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.808319 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.808727 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.808835 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.812468 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:40.812484 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:40.812500 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:40.812537 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:40.816002 (+ 3465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:40.816886 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:40.816893 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:40.816896 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:40.817016 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:40.817455 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:40.817459 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:40.817461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:40.817854 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:40.818084 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:40.818526 (+ 442us) server_negotiation.cc:299] Negotiation successful
0810 14:48:40.818756 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260810 14:48:40.819907 12509 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:48:40.858724 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.858181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46201 (local address 127.0.0.1:52254)
0810 14:48:40.858428 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:40.858546 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260810 14:48:40.860116 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.859586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46201 (local address 127.0.0.1:52270)
0810 14:48:40.859838 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:40.859928 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142}
I20260810 14:48:40.910892 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 12370
I20260810 14:48:40.920856 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.920235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46201 (local address 127.0.0.1:52286)
0810 14:48:40.920517 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:40.920636 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90}
I20260810 14:48:40.922200 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.921718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46201 (local address 127.0.0.1:52290)
0810 14:48:40.921912 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:40.922020 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260810 14:48:40.936353 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 12519
I20260810 14:48:40.952966 12684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:40.937330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60072 (local address 127.24.215.131:39053)
0810 14:48:40.937507 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 14:48:40.937512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:48:40.937533 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:48:40.937635 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:48:40.937639 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:48:40.937721 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:48:40.937801 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:48:40.939826 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.940648 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.942607 (+ 1959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:48:40.943061 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:48:40.943240 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:48:40.946701 (+ 3461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:48:40.946713 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:48:40.946729 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:48:40.946764 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:48:40.949857 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:40.950813 (+ 956us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:40.950820 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:40.950823 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:40.950981 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:48:40.951459 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:48:40.951471 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:48:40.951473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:48:40.951836 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:48:40.952085 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0810 14:48:40.952451 (+ 366us) server_negotiation.cc:299] Negotiation successful
0810 14:48:40.952623 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 14:48:40.953790 12662 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:48:41.002940 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:41.002342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46201 (local address 127.0.0.1:52304)
0810 14:48:41.002636 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:41.002756 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129}
I20260810 14:48:41.004186 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:41.003690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46201 (local address 127.0.0.1:52308)
0810 14:48:41.003915 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:41.004007 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105}
I20260810 14:48:41.080634 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 12519
I20260810 14:48:41.104127 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 12117
I20260810 14:48:41.105345 12189 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:48:41.219950 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 12117
I20260810 14:48:41.220059 12720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:48:41.219351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:46201 (local address 127.0.0.1:52324)
0810 14:48:41.219597 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0810 14:48:41.219793 (+ 196us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:46201: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260810 14:49:01.247735 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:33161
--webserver_interface=127.24.215.190
--webserver_port=36225
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:35623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:33161
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:01.589756 12725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:01.590186 12725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:01.590342 12725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:01.598727 12725 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260810 14:49:01.598829 12725 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260810 14:49:01.598866 12725 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260810 14:49:01.598912 12725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:49:01.598973 12725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:01.599048 12725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:01.599139 12725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:49:01.599208 12725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:49:01.609280 12725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:35623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:33161
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:33161
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=36225
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:01.610527 12725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:01.611903 12725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:01.624138 12730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:01.624176 12731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:01.624341 12725 server_base.cc:1034] running on GCE node
W20260810 14:49:01.624118 12733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:01.625550 12725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:01.626916 12725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:01.628170 12725 hybrid_clock.cc:648] HybridClock initialized: now 1786373341628094 us; error 55 us; skew 500 ppm
I20260810 14:49:01.628507 12725 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:01 dist-test-slave-vkjp krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373341, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:01.631716 12725 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:49:01.633222 12725 webserver.cc:460] Webserver started at http://127.24.215.190:36225/ using document root <none> and password file <none>
I20260810 14:49:01.633936 12725 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:01.634075 12725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:01.639599 12725 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260810 14:49:01.642481 12740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:01.643776 12725 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:49:01.643967 12725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "447594d189964ab78597f51ccd72df22"
format_stamp: "Formatted at 2026-08-10 14:48:15 on dist-test-slave-vkjp"
server_key: "9dbc6f37b65d8c44c24cf278d22b2f30"
server_key_iv: "d6d6ed141e80b2a1d17be03d771cc78a"
server_key_version: "encryptionkey@0"
I20260810 14:49:01.644958 12725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:01.662007 12725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:01.665846 12725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:01.666146 12725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:01.685089 12725 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:33161
I20260810 14:49:01.685124 12803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:33161 every 8 connection(s)
I20260810 14:49:01.686614 12725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:49:01.693748 12804 sys_catalog.cc:263] Verifying existing consensus state
I20260810 14:49:01.696271 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 12725
I20260810 14:49:01.697083 12804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Bootstrap starting.
I20260810 14:49:01.697136 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:46201
--local_ip_for_outbound_sockets=127.24.215.129
--tserver_master_addrs=127.24.215.190:33161
--webserver_port=39855
--webserver_interface=127.24.215.129
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:35623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:49:01.724810 12804 log.cc:824] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:01.741009 12804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:49:01.741629 12804 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Bootstrap complete.
I20260810 14:49:01.751693 12804 raft_consensus.cc:348] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } }
I20260810 14:49:01.752600 12804 raft_consensus.cc:729] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 447594d189964ab78597f51ccd72df22, State: Initialized, Role: FOLLOWER
I20260810 14:49:01.753265 12804 consensus_queue.cc:260] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } }
I20260810 14:49:01.753521 12804 raft_consensus.cc:388] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:49:01.753645 12804 raft_consensus.cc:482] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:49:01.753844 12804 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 2 FOLLOWER]: Advancing to term 3
I20260810 14:49:01.757843 12804 raft_consensus.cc:504] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } }
I20260810 14:49:01.758345 12804 leader_election.cc:302] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 447594d189964ab78597f51ccd72df22; no voters:
I20260810 14:49:01.759065 12804 leader_election.cc:288] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260810 14:49:01.759250 12809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 3 FOLLOWER]: Leader election won for term 3
I20260810 14:49:01.760499 12809 raft_consensus.cc:686] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [term 3 LEADER]: Becoming Leader. State: Replica: 447594d189964ab78597f51ccd72df22, State: Running, Role: LEADER
I20260810 14:49:01.761181 12809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } }
I20260810 14:49:01.761443 12804 sys_catalog.cc:564] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:49:01.763785 12812 sys_catalog.cc:455] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 447594d189964ab78597f51ccd72df22. Latest consensus state: current_term: 3 leader_uuid: "447594d189964ab78597f51ccd72df22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } } }
I20260810 14:49:01.764158 12812 sys_catalog.cc:458] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:01.764592 12811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "447594d189964ab78597f51ccd72df22" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "447594d189964ab78597f51ccd72df22" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 33161 } } }
I20260810 14:49:01.764864 12811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:01.777098 12817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:49:01.786156 12817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:49:01.787783 12817 catalog_manager.cc:1259] Loaded cluster ID: 244d1fe036a84d9dae6fd7f3c4d41241
I20260810 14:49:01.787987 12817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:49:01.792528 12817 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:49:01.794540 12817 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Loaded TSK: 0
I20260810 14:49:01.794788 12817 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 447594d189964ab78597f51ccd72df22: Loaded TSK: 1
F20260810 14:49:01.796238 12817 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f92383539ad google::LogMessage::Fail() at ??:0
@ 0x7f923835688a google::LogMessage::SendToLog() at ??:0
@ 0x7f92383534de google::LogMessage::Flush() at ??:0
@ 0x7f9238354a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f923c0b77d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f923c0a673e kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f924dd821c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f924dd81783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f924de0d2a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f924de0d0c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f924de575e9 std::function<>::operator()() at ??:0
@ 0x7f924dd85cdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f924dd83c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f924de0b610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f924de0b439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f923af9f345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f923afa291c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f923afa2799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f923af7deaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f923e2576db start_thread at ??:0
@ 0x7f92371ed61f clone at ??:0
W20260810 14:49:02.098987 12806 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:02.099349 12806 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:02.099537 12806 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:02.107059 12806 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:02.107259 12806 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:02.107450 12806 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:49:02.117550 12806 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:35623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:46201
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=39855
--tserver_master_addrs=127.24.215.190:33161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:02.118863 12806 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:02.120275 12806 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:02.133981 12858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:02.134003 12857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:02.134090 12860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:02.135027 12806 server_base.cc:1034] running on GCE node
I20260810 14:49:02.135664 12806 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:02.136545 12806 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:02.137749 12806 hybrid_clock.cc:648] HybridClock initialized: now 1786373342137703 us; error 31 us; skew 500 ppm
I20260810 14:49:02.138132 12806 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:02 dist-test-slave-vkjp krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373342, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:02.141301 12806 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:49:02.142728 12806 webserver.cc:460] Webserver started at http://127.24.215.129:39855/ using document root <none> and password file <none>
I20260810 14:49:02.143540 12806 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:02.143697 12806 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:02.149150 12806 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260810 14:49:02.152029 12868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:02.153366 12806 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:49:02.153554 12806 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "5fd5ddb374244f2dbe0308b439d9af6b"
format_stamp: "Formatted at 2026-08-10 14:48:15 on dist-test-slave-vkjp"
server_key: "3108b40e065ee9ca18cb14df0fe0ae73"
server_key_iv: "b8bb86477f6f63a659084e4a16aa06e6"
server_key_version: "encryptionkey@0"
I20260810 14:49:02.154567 12806 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:02.167757 12806 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:02.171772 12806 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:02.172075 12806 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:02.173094 12806 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:02.174907 12876 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260810 14:49:02.180536 12806 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260810 14:49:02.180681 12806 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260810 14:49:02.180830 12806 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260810 14:49:02.184015 12806 ts_tablet_manager.cc:610] Registered 1 tablets
I20260810 14:49:02.184124 12806 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.003s sys 0.000s
I20260810 14:49:02.184455 12876 tablet_bootstrap.cc:492] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b: Bootstrap starting.
I20260810 14:49:02.212822 12876 log.cc:824] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:02.225518 12806 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:46201
I20260810 14:49:02.225581 13003 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:46201 every 8 connection(s)
I20260810 14:49:02.226279 12876 tablet_bootstrap.cc:492] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260810 14:49:02.226837 12876 tablet_bootstrap.cc:492] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b: Bootstrap complete.
I20260810 14:49:02.226833 12806 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:49:02.228585 12876 ts_tablet_manager.cc:1397] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b: Time spent bootstrapping tablet: real 0.044s user 0.027s sys 0.015s
I20260810 14:49:02.229863 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 12806
I20260810 14:49:02.230618 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:37741
--local_ip_for_outbound_sockets=127.24.215.130
--tserver_master_addrs=127.24.215.190:33161
--webserver_port=34093
--webserver_interface=127.24.215.130
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:35623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:49:02.232686 13006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:02.229968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:36175)
0810 14:49:02.231037 (+ 1069us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:02.231401 (+ 364us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":723,"thread_start_us":458,"threads_started":1}
W20260810 14:49:02.233457 13004 heartbeater.cc:644] Failed to heartbeat to 127.24.215.190:33161 (0 consecutive failures): Network error: Failed to ping master at 127.24.215.190:33161: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
I20260810 14:49:02.234556 13006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:02.234076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:45319)
0810 14:49:02.234279 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:02.234382 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 14:49:02.235615 13006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:02.235180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:51033)
0810 14:49:02.235336 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:02.235436 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 14:49:02.235913 13004 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 14:49:02.237349 12876 raft_consensus.cc:348] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } }
I20260810 14:49:02.238210 12876 raft_consensus.cc:729] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5fd5ddb374244f2dbe0308b439d9af6b, State: Initialized, Role: FOLLOWER
I20260810 14:49:02.238732 12876 consensus_queue.cc:260] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } }
I20260810 14:49:02.240351 13006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:02.240002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:48667)
0810 14:49:02.240129 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:02.240209 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260810 14:49:02.240448 12876 ts_tablet_manager.cc:1428] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260810 14:49:02.241365 13006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:02.241042 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:40371)
0810 14:49:02.241188 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:02.241261 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 14:49:02.579797 13008 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:02.580152 13008 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:02.580389 13008 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:02.587968 13008 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:02.588115 13008 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:02.588336 13008 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:49:02.598613 13008 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:35623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:37741
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=34093
--tserver_master_addrs=127.24.215.190:33161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:02.600133 13008 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:02.601686 13008 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:02.615620 13014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:02.615667 13016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:02.615695 13013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:02.616483 13008 server_base.cc:1034] running on GCE node
I20260810 14:49:02.617180 13008 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:02.618153 13008 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:02.619401 13008 hybrid_clock.cc:648] HybridClock initialized: now 1786373342619350 us; error 57 us; skew 500 ppm
I20260810 14:49:02.619740 13008 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:02 dist-test-slave-vkjp krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373342, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:02.622961 13008 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:49:02.624544 13008 webserver.cc:460] Webserver started at http://127.24.215.130:34093/ using document root <none> and password file <none>
I20260810 14:49:02.625298 13008 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:02.625465 13008 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:02.630903 13008 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:49:02.633605 13026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:02.634943 13008 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:49:02.635202 13008 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "78ead0c12f3b475dbe557b7e16104bec"
format_stamp: "Formatted at 2026-08-10 14:48:16 on dist-test-slave-vkjp"
server_key: "6cd86e01202f3d6617963dd6631dec24"
server_key_iv: "814b3e8723fa6bb8a366d35cbb08ec9a"
server_key_version: "encryptionkey@0"
I20260810 14:49:02.636217 13008 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:02.649180 13008 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:02.653263 13008 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:02.653566 13008 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:02.654517 13008 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:02.656068 13008 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:02.656211 13008 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:02.656337 13008 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:02.656433 13008 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:02.690310 13008 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:37741
I20260810 14:49:02.690398 13151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:37741 every 8 connection(s)
I20260810 14:49:02.691915 13008 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:49:02.696823 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:02.694972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:50597)
0810 14:49:02.695728 (+ 756us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:02.695953 (+ 225us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":299,"threads_started":1}
W20260810 14:49:02.697391 13152 heartbeater.cc:644] Failed to heartbeat to 127.24.215.190:33161 (0 consecutive failures): Network error: Failed to ping master at 127.24.215.190:33161: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
I20260810 14:49:02.698233 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:02.697924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:60079)
0810 14:49:02.698063 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:02.698127 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 14:49:02.699003 13154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:02.698714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:34471)
0810 14:49:02.698827 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:02.698895 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260810 14:49:02.699237 13152 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 14:49:02.701687 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 13008
I20260810 14:49:02.702399 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:39053
--local_ip_for_outbound_sockets=127.24.215.131
--tserver_master_addrs=127.24.215.190:33161
--webserver_port=39063
--webserver_interface=127.24.215.131
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:35623
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:03.036674 13155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:03.036983 13155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:03.037077 13155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:03.044286 13155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:03.044394 13155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:03.044505 13155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:49:03.053849 13155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:35623
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:39053
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=39063
--tserver_master_addrs=127.24.215.190:33161
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:03.055068 13155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:03.056351 13155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:03.069731 13160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:03.069779 13163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:03.070093 13161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:03.070798 13155 server_base.cc:1034] running on GCE node
I20260810 14:49:03.071488 13155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:03.072394 13155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:03.073587 13155 hybrid_clock.cc:648] HybridClock initialized: now 1786373343073532 us; error 53 us; skew 500 ppm
I20260810 14:49:03.073920 13155 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:03 dist-test-slave-vkjp krb5kdc[11487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373343, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:03.077453 13155 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:49:03.078857 13155 webserver.cc:460] Webserver started at http://127.24.215.131:39063/ using document root <none> and password file <none>
I20260810 14:49:03.079589 13155 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:03.079695 13155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:03.084900 13155 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260810 14:49:03.087512 13171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:03.088828 13155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:49:03.089003 13155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "0ec912c539e74652906603d968389e37"
format_stamp: "Formatted at 2026-08-10 14:48:16 on dist-test-slave-vkjp"
server_key: "28dca4ddeca3a8004b1e0b2440b1cb5e"
server_key_iv: "b3398cd12aa833b3bf51200d2c744969"
server_key_version: "encryptionkey@0"
I20260810 14:49:03.089955 13155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:03.106757 13155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:03.110903 13155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:03.111236 13155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:03.112196 13155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:03.113461 13155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:03.113582 13155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:03.113710 13155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:03.113795 13155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:03.147320 13155 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:39053
I20260810 14:49:03.147362 13298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:39053 every 8 connection(s)
I20260810 14:49:03.148782 13155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:49:03.149616 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 13155
I20260810 14:49:03.154531 13302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:03.152822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:56333)
0810 14:49:03.153517 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:03.153730 (+ 213us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":269,"threads_started":1}
W20260810 14:49:03.155257 13299 heartbeater.cc:644] Failed to heartbeat to 127.24.215.190:33161 (0 consecutive failures): Network error: Failed to ping master at 127.24.215.190:33161: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
I20260810 14:49:03.156010 13302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:03.155706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:38921)
0810 14:49:03.155849 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:03.155916 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260810 14:49:03.156809 13302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:03.156499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:40675)
0810 14:49:03.156635 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:03.156711 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 14:49:03.157030 13299 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260810 14:49:03.242857 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:03.242112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:45013)
0810 14:49:03.242567 (+ 455us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:03.242656 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":223,"threads_started":1}
I20260810 14:49:03.675166 13304 raft_consensus.cc:482] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:49:03.675491 13304 raft_consensus.cc:504] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } }
I20260810 14:49:03.677354 13304 leader_election.cc:288] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec912c539e74652906603d968389e37 (127.24.215.131:39053), 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741)
Aug 10 14:49:03 dist-test-slave-vkjp krb5kdc[11487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373342, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
Aug 10 14:49:03 dist-test-slave-vkjp krb5kdc[11487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373342, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:49:03.699949 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:03.677690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:39053 (local address 127.24.215.129:58575)
0810 14:49:03.677952 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:03.678009 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:03.678685 (+ 676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:03.682546 (+ 3861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:03.682558 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:03.683464 (+ 906us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:03.684177 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:03.684197 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:03.685724 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:03.685728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:03.687069 (+ 1341us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:03.687129 (+ 60us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:03.688160 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:03.688167 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:03.688421 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:03.689210 (+ 789us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:49:03.689233 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:03.692591 (+ 3358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:03.696485 (+ 3894us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:03.696494 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:03.696508 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:03.696923 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:03.697362 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:03.697378 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:03.697381 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:03.697598 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:03.698395 (+ 797us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:03.698403 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:03.698699 (+ 296us) client_negotiation.cc:769] Sending connection context
0810 14:49:03.698983 (+ 284us) client_negotiation.cc:240] Negotiation successful
0810 14:49:03.699518 (+ 535us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"mutex_wait_us":60,"spinlock_wait_cycles":13696}
I20260810 14:49:03.700035 13306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:03.678101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:58575 (local address 127.24.215.131:39053)
0810 14:49:03.679925 (+ 1824us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:03.679946 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:03.680010 (+ 64us) server_negotiation.cc:407] Connection header received
0810 14:49:03.681658 (+ 1648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:03.681668 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:03.682074 (+ 406us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:03.682420 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:03.684385 (+ 1965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:03.685536 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:03.687260 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:03.687900 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:03.688117 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:03.692846 (+ 4729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:03.692854 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:03.692872 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:03.692915 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:03.696323 (+ 3408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:03.697080 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:03.697087 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:03.697093 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:03.697243 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:03.697732 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:03.697737 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:03.697739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:03.698236 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:03.698441 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:03.699231 (+ 790us) server_negotiation.cc:299] Negotiation successful
0810 14:49:03.699622 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1587,"thread_start_us":1451,"threads_started":1}
I20260810 14:49:03.700639 13307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:03.678082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:58085 (local address 127.24.215.130:37741)
0810 14:49:03.680220 (+ 2138us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:03.680238 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:03.680586 (+ 348us) server_negotiation.cc:407] Connection header received
0810 14:49:03.682176 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:03.682185 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:03.682507 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:03.682799 (+ 292us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:03.684398 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:03.685515 (+ 1117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:03.687249 (+ 1734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:03.687901 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:03.688119 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:03.692615 (+ 4496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:03.692627 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:03.692644 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:03.692689 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:03.697012 (+ 4323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:03.697815 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:03.697820 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:03.697825 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:03.697959 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:03.698363 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:03.698369 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:03.698378 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:03.698824 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:03.699028 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:03.699752 (+ 724us) server_negotiation.cc:299] Negotiation successful
0810 14:49:03.700208 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1964,"thread_start_us":1836,"threads_started":1}
I20260810 14:49:03.700966 13305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:03.677829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:37741 (local address 127.24.215.129:58085)
0810 14:49:03.680459 (+ 2630us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:03.680473 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:03.680631 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:03.682767 (+ 2136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:03.682772 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:03.683524 (+ 752us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:03.684177 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:03.684198 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:03.685719 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:03.685723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:03.687061 (+ 1338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:03.687074 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:03.688128 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:03.688135 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:03.688424 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:03.689265 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:49:03.689283 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:03.692265 (+ 2982us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:03.697170 (+ 4905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:03.697175 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:03.697178 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:03.697655 (+ 477us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:03.698080 (+ 425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:03.698084 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:03.698088 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:03.698241 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:03.698965 (+ 724us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:03.698970 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:03.699183 (+ 213us) client_negotiation.cc:769] Sending connection context
0810 14:49:03.699359 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 14:49:03.700558 (+ 1199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2526,"mutex_wait_us":1099,"spinlock_wait_cycles":13056,"thread_start_us":2258,"threads_started":1}
I20260810 14:49:03.701561 13308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:03.699762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:49305)
0810 14:49:03.700668 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:03.701288 (+ 620us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":766,"thread_start_us":1310,"threads_started":1}
W20260810 14:49:03.704032 12873 leader_election.cc:341] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec912c539e74652906603d968389e37 (127.24.215.131:39053): Not found: Tablet not found: 5f46f335050845d5b4e51c33e66e1d23
W20260810 14:49:03.704550 12871 leader_election.cc:341] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741): Not found: Tablet not found: 5f46f335050845d5b4e51c33e66e1d23
I20260810 14:49:03.704696 12871 leader_election.cc:302] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fd5ddb374244f2dbe0308b439d9af6b; no voters: 0ec912c539e74652906603d968389e37, 78ead0c12f3b475dbe557b7e16104bec
I20260810 14:49:03.705191 13304 raft_consensus.cc:2726] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:49:04.158591 13309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:04.157675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:41845)
0810 14:49:04.158204 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:04.158349 (+ 145us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":290,"threads_started":1}
I20260810 14:49:04.244654 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:04.243808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:57885)
0810 14:49:04.244354 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:04.244450 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":300,"threads_started":1}
I20260810 14:49:04.704555 13311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:04.703844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:42387)
0810 14:49:04.704303 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:04.704391 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":239,"threads_started":1}
I20260810 14:49:05.160171 13312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:05.159528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:37445)
0810 14:49:05.159928 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:05.160020 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":202,"threads_started":1}
I20260810 14:49:05.246316 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:05.245537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:33745)
0810 14:49:05.246033 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:05.246114 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":223,"threads_started":1}
I20260810 14:49:05.645057 13314 raft_consensus.cc:482] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:49:05.645293 13314 raft_consensus.cc:504] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } }
I20260810 14:49:05.645964 13314 leader_election.cc:288] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec912c539e74652906603d968389e37 (127.24.215.131:39053), 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741)
W20260810 14:49:05.647004 12871 leader_election.cc:341] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741): Not found: Tablet not found: 5f46f335050845d5b4e51c33e66e1d23
W20260810 14:49:05.647212 12873 leader_election.cc:341] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec912c539e74652906603d968389e37 (127.24.215.131:39053): Not found: Tablet not found: 5f46f335050845d5b4e51c33e66e1d23
I20260810 14:49:05.647380 12873 leader_election.cc:302] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fd5ddb374244f2dbe0308b439d9af6b; no voters: 0ec912c539e74652906603d968389e37, 78ead0c12f3b475dbe557b7e16104bec
I20260810 14:49:05.647661 13314 raft_consensus.cc:2726] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:49:05.706248 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:05.705512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:46061)
0810 14:49:05.705997 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:05.706088 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":249,"threads_started":1}
I20260810 14:49:06.161799 13316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:06.161071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:44157)
0810 14:49:06.161547 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:06.161639 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":251,"threads_started":1}
I20260810 14:49:06.247949 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:06.247247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:50153)
0810 14:49:06.247688 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:06.247781 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":233,"threads_started":1}
I20260810 14:49:06.708127 13321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:06.707365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:52617)
0810 14:49:06.707832 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:06.707926 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":224,"threads_started":1}
I20260810 14:49:07.163506 13322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:07.162725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:53011)
0810 14:49:07.163232 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:07.163313 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":234,"threads_started":1}
I20260810 14:49:07.249864 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:07.249084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:45151)
0810 14:49:07.249574 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:07.249661 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":263,"threads_started":1}
I20260810 14:49:07.709869 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:07.709119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:36533)
0810 14:49:07.709591 (+ 472us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:07.709680 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":257,"threads_started":1}
I20260810 14:49:07.811721 13325 raft_consensus.cc:482] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:49:07.812011 13325 raft_consensus.cc:504] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } }
I20260810 14:49:07.812832 13325 leader_election.cc:288] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec912c539e74652906603d968389e37 (127.24.215.131:39053), 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741)
W20260810 14:49:07.813737 12871 leader_election.cc:341] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741): Not found: Tablet not found: 5f46f335050845d5b4e51c33e66e1d23
W20260810 14:49:07.813948 12873 leader_election.cc:341] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec912c539e74652906603d968389e37 (127.24.215.131:39053): Not found: Tablet not found: 5f46f335050845d5b4e51c33e66e1d23
I20260810 14:49:07.814116 12873 leader_election.cc:302] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fd5ddb374244f2dbe0308b439d9af6b; no voters: 0ec912c539e74652906603d968389e37, 78ead0c12f3b475dbe557b7e16104bec
I20260810 14:49:07.814412 13325 raft_consensus.cc:2726] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:49:08.165117 13326 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:08.164442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:52997)
0810 14:49:08.164866 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:08.164950 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":229,"threads_started":1}
I20260810 14:49:08.252171 13328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:08.251193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:59421)
0810 14:49:08.251838 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:08.251974 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":513,"thread_start_us":345,"threads_started":1}
I20260810 14:49:08.711876 13329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:08.710937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:34173)
0810 14:49:08.711556 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:08.711643 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":375,"threads_started":1}
I20260810 14:49:09.166792 13330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:09.166068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:47923)
0810 14:49:09.166536 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:09.166617 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":256,"threads_started":1}
I20260810 14:49:09.253727 13331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:09.253032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:54559)
0810 14:49:09.253462 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:09.253540 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":238,"threads_started":1}
I20260810 14:49:09.713709 13332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:09.712888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:42405)
0810 14:49:09.713431 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:09.713517 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":405,"thread_start_us":227,"threads_started":1}
I20260810 14:49:09.793423 13333 raft_consensus.cc:482] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:49:09.793668 13333 raft_consensus.cc:504] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } }
I20260810 14:49:09.794423 13333 leader_election.cc:288] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec912c539e74652906603d968389e37 (127.24.215.131:39053), 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741)
W20260810 14:49:09.795611 12873 leader_election.cc:341] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec912c539e74652906603d968389e37 (127.24.215.131:39053): Not found: Tablet not found: 5f46f335050845d5b4e51c33e66e1d23
W20260810 14:49:09.795857 12871 leader_election.cc:341] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741): Not found: Tablet not found: 5f46f335050845d5b4e51c33e66e1d23
I20260810 14:49:09.796022 12871 leader_election.cc:302] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fd5ddb374244f2dbe0308b439d9af6b; no voters: 0ec912c539e74652906603d968389e37, 78ead0c12f3b475dbe557b7e16104bec
I20260810 14:49:09.796373 13333 raft_consensus.cc:2726] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:49:10.168642 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:10.167794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:33105)
0810 14:49:10.168369 (+ 575us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:10.168456 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":267,"threads_started":1}
I20260810 14:49:10.255753 13336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:10.254637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:33039)
0810 14:49:10.255426 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:10.255544 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":463,"threads_started":1}
I20260810 14:49:10.715411 13337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:10.714655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:51267)
0810 14:49:10.715135 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:10.715223 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":225,"threads_started":1}
I20260810 14:49:11.170297 13338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:11.169560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:51387)
0810 14:49:11.170008 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:11.170092 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":237,"threads_started":1}
I20260810 14:49:11.257433 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:11.256644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:33211)
0810 14:49:11.257180 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:11.257263 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":326,"threads_started":1}
I20260810 14:49:11.717237 13340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:11.716349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:35485)
0810 14:49:11.716940 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:11.717063 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":246,"threads_started":1}
I20260810 14:49:12.172416 13341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:12.171433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:49045)
0810 14:49:12.172184 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:12.172264 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":648,"thread_start_us":363,"threads_started":1}
I20260810 14:49:12.259470 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:12.258530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:45061)
0810 14:49:12.258992 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:12.259146 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":247,"threads_started":1}
I20260810 14:49:12.718971 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:12.718193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:37545)
0810 14:49:12.718679 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:12.718770 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":264,"threads_started":1}
I20260810 14:49:13.174237 13344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:13.173430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:53229)
0810 14:49:13.173951 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:13.174053 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":293,"threads_started":1}
I20260810 14:49:13.261153 13345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:13.260409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:50205)
0810 14:49:13.260863 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:13.260956 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":236,"threads_started":1}
I20260810 14:49:13.720722 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:13.719947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:37985)
0810 14:49:13.720445 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:13.720535 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":234,"threads_started":1}
I20260810 14:49:14.175925 13347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:14.175242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:54097)
0810 14:49:14.175689 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:14.175771 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":217,"threads_started":1}
I20260810 14:49:14.262694 13348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:14.262012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:48681)
0810 14:49:14.262452 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:14.262532 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":231,"threads_started":1}
I20260810 14:49:14.441320 13349 raft_consensus.cc:482] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:49:14.441623 13349 raft_consensus.cc:504] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5fd5ddb374244f2dbe0308b439d9af6b" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 46201 } } peers { permanent_uuid: "0ec912c539e74652906603d968389e37" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 39053 } } peers { permanent_uuid: "78ead0c12f3b475dbe557b7e16104bec" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37741 } }
I20260810 14:49:14.442324 13349 leader_election.cc:288] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 0ec912c539e74652906603d968389e37 (127.24.215.131:39053), 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741)
W20260810 14:49:14.443353 12873 leader_election.cc:341] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 0ec912c539e74652906603d968389e37 (127.24.215.131:39053): Not found: Tablet not found: 5f46f335050845d5b4e51c33e66e1d23
W20260810 14:49:14.443562 12871 leader_election.cc:341] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 78ead0c12f3b475dbe557b7e16104bec (127.24.215.130:37741): Not found: Tablet not found: 5f46f335050845d5b4e51c33e66e1d23
I20260810 14:49:14.443674 12871 leader_election.cc:302] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5fd5ddb374244f2dbe0308b439d9af6b; no voters: 0ec912c539e74652906603d968389e37, 78ead0c12f3b475dbe557b7e16104bec
I20260810 14:49:14.443964 13349 raft_consensus.cc:2726] T 5f46f335050845d5b4e51c33e66e1d23 P 5fd5ddb374244f2dbe0308b439d9af6b [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260810 14:49:14.722558 13350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:14.721779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:47437)
0810 14:49:14.722260 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:14.722384 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":247,"threads_started":1}
I20260810 14:49:15.177767 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:15.176937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:55039)
0810 14:49:15.177474 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:15.177559 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":298,"threads_started":1}
I20260810 14:49:15.264673 13352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:15.263848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:56173)
0810 14:49:15.264348 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:15.264466 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":377,"thread_start_us":242,"threads_started":1}
I20260810 14:49:15.724247 13353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:15.723532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:33959)
0810 14:49:15.723991 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:15.724081 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":243,"threads_started":1}
I20260810 14:49:16.179613 13354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:16.178797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:36169)
0810 14:49:16.179356 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:16.179449 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":238,"threads_started":1}
I20260810 14:49:16.266325 13356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:16.265600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:37841)
0810 14:49:16.266039 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:16.266117 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":226,"threads_started":1}
I20260810 14:49:16.725926 13357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:16.725181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:44913)
0810 14:49:16.725644 (+ 463us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:16.725726 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":227,"threads_started":1}
I20260810 14:49:17.181252 13358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:17.180508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:51709)
0810 14:49:17.180962 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:17.181056 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":230,"threads_started":1}
I20260810 14:49:17.268255 13359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:17.267536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.129:43535)
0810 14:49:17.267997 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:17.268087 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":226,"threads_started":1}
I20260810 14:49:17.727792 13360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:17.727021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.130:52717)
0810 14:49:17.727505 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:17.727629 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":267,"threads_started":1}
I20260810 14:49:18.161755 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 12806
I20260810 14:49:18.181679 13361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:18.162818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40490 (local address 127.24.215.129:46201)
0810 14:49:18.163473 (+ 655us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:18.163485 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:18.163655 (+ 170us) server_negotiation.cc:407] Connection header received
0810 14:49:18.163907 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:18.163919 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:18.164244 (+ 325us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:18.164439 (+ 195us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:18.165863 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:18.166794 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:18.168178 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:18.168721 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:18.168824 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:18.175860 (+ 7036us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:18.175874 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:18.175890 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:18.175931 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:18.178835 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:18.179705 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:18.179712 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:18.179714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:18.179860 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:18.180260 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:18.180263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:18.180265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:18.180604 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:18.180846 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:18.181182 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 14:49:18.181386 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":269,"threads_started":1}
I20260810 14:49:18.183073 12999 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:18.183341 13362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:18.182437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:33161 (local address 127.24.215.131:34301)
0810 14:49:18.182952 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:18.183128 (+ 176us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.190:33161: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":397,"thread_start_us":281,"threads_started":1}
I20260810 14:49:18.272620 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 12806
I20260810 14:49:18.294798 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 13008
I20260810 14:49:18.308660 13365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:18.295701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36608 (local address 127.24.215.130:37741)
0810 14:49:18.296234 (+ 533us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:18.296239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:18.296255 (+ 16us) server_negotiation.cc:407] Connection header received
0810 14:49:18.296315 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:18.296319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:18.296386 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:18.296481 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:18.297799 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:18.298603 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:18.299903 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:18.300318 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:18.300419 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:18.303441 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:18.303446 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:18.303450 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:18.303481 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:18.306020 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:18.306629 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:18.306635 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:18.306637 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:18.306789 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:18.307407 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:18.307411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:18.307414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:18.307692 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:18.307908 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:18.308161 (+ 253us) server_negotiation.cc:299] Negotiation successful
0810 14:49:18.308348 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":444,"thread_start_us":306,"threads_started":1}
I20260810 14:49:18.309379 13147 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:18.394496 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 13008
I20260810 14:49:18.415053 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 13155
I20260810 14:49:18.429171 13367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:18.415964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53990 (local address 127.24.215.131:39053)
0810 14:49:18.416504 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:18.416510 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:18.416531 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:49:18.416592 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:18.416596 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:18.416674 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:18.416769 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:18.418190 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:18.419013 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:18.420353 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:18.420779 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:18.420887 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:18.423661 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:18.423666 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:18.423670 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:18.423706 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:18.426329 (+ 2623us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:18.427118 (+ 789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:18.427124 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:18.427128 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:18.427288 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:18.427693 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:18.427698 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:18.427702 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:18.427974 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:18.428257 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:18.428572 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 14:49:18.428800 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":411,"thread_start_us":263,"threads_started":1}
I20260810 14:49:18.429862 13294 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:18.514653 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 13155
2026-08-10T14:49:18Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65109 ms)
[----------] 30 tests from SecurityITest (306882 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:49:18 dist-test-slave-vkjp krb5kdc[13371](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:49:18 dist-test-slave-vkjp krb5kdc[13371](info): set up 2 sockets
Aug 10 14:49:18 dist-test-slave-vkjp krb5kdc[13371](info): commencing operation
krb5kdc: starting...
W20260810 14:49:19.728556 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:49:19 dist-test-slave-vkjp krb5kdc[13371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:49:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:49:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:49:20.022557 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:44917
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:44917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:20.363945 13385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:20.364267 13385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:20.364439 13385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:20.371762 13385 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:49:20.371905 13385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:20.372013 13385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:20.372113 13385 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:49:20.372198 13385 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:49:20.383721 13385 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:44917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:44917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:20.385138 13385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:20.386473 13385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:20.399142 13391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:20.399276 13390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:20.399247 13393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:20.399571 13385 server_base.cc:1034] running on GCE node
I20260810 14:49:20.400617 13385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:20.402040 13385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:20.403290 13385 hybrid_clock.cc:648] HybridClock initialized: now 1786373360403219 us; error 45 us; skew 500 ppm
I20260810 14:49:20.403728 13385 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:20 dist-test-slave-vkjp krb5kdc[13371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373360, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:20.408281 13385 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:49:20.410815 13385 webserver.cc:460] Webserver started at http://127.24.215.190:44809/ using document root <none> and password file <none>
I20260810 14:49:20.411855 13385 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:20.412014 13385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:20.412412 13385 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:20.417092 13385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "1286f8c44fa94dd58cb6d29ac36e229a"
format_stamp: "Formatted at 2026-08-10 14:49:20 on dist-test-slave-vkjp"
server_key: "32a68bd464ece15b2a793a91eccfa410"
server_key_iv: "2c9ebbf2058fa2a112efef269a09a72d"
server_key_version: "encryptionkey@0"
I20260810 14:49:20.418161 13385 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "1286f8c44fa94dd58cb6d29ac36e229a"
format_stamp: "Formatted at 2026-08-10 14:49:20 on dist-test-slave-vkjp"
server_key: "32a68bd464ece15b2a793a91eccfa410"
server_key_iv: "2c9ebbf2058fa2a112efef269a09a72d"
server_key_version: "encryptionkey@0"
I20260810 14:49:20.425098 13385 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.008s sys 0.000s
I20260810 14:49:20.430210 13401 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:20.431965 13385 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260810 14:49:20.432164 13385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "1286f8c44fa94dd58cb6d29ac36e229a"
format_stamp: "Formatted at 2026-08-10 14:49:20 on dist-test-slave-vkjp"
server_key: "32a68bd464ece15b2a793a91eccfa410"
server_key_iv: "2c9ebbf2058fa2a112efef269a09a72d"
server_key_version: "encryptionkey@0"
I20260810 14:49:20.432446 13385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:20.456336 13385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:20.460954 13385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:20.461215 13385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:20.480185 13385 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:44917
I20260810 14:49:20.480207 13460 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:44917 every 8 connection(s)
I20260810 14:49:20.481618 13385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:49:20.482311 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 13385
I20260810 14:49:20.482551 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:49:20.483412 25438 external_mini_cluster.cc:1428] Setting key 188ca1fe4ec6cb71005310bbc6e58e3a
I20260810 14:49:20.486222 13461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:20.497036 13461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a: Bootstrap starting.
Aug 10 14:49:20 dist-test-slave-vkjp krb5kdc[13371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:49:20.501035 13461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:20.502106 13461 log.cc:824] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:20.505038 13461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a: No bootstrap required, opened a new log
I20260810 14:49:20.506348 13466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:20.485501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54736 (local address 127.24.215.190:44917)
0810 14:49:20.486309 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:20.486326 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:20.486382 (+ 56us) server_negotiation.cc:407] Connection header received
0810 14:49:20.488131 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:20.488154 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:20.488528 (+ 374us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:20.488987 (+ 459us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:20.490549 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:20.491956 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:20.493334 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:20.494059 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:20.494265 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:20.498985 (+ 4720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:20.498996 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:20.499011 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:20.499049 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:20.502209 (+ 3160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:20.502966 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:20.502977 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:20.503012 (+ 35us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:20.503245 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:20.503665 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:20.503671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:20.503674 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:20.504184 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:20.504517 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:20.504940 (+ 423us) server_negotiation.cc:299] Negotiation successful
0810 14:49:20.505390 (+ 450us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":350,"threads_started":1}
I20260810 14:49:20.513937 13461 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1286f8c44fa94dd58cb6d29ac36e229a" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44917 } }
I20260810 14:49:20.514315 13461 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:20.514428 13461 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1286f8c44fa94dd58cb6d29ac36e229a, State: Initialized, Role: FOLLOWER
I20260810 14:49:20.514971 13461 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1286f8c44fa94dd58cb6d29ac36e229a" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44917 } }
I20260810 14:49:20.515281 13461 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:49:20.515434 13461 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:49:20.515621 13461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:20.517817 13461 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1286f8c44fa94dd58cb6d29ac36e229a" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44917 } }
I20260810 14:49:20.518381 13461 leader_election.cc:302] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1286f8c44fa94dd58cb6d29ac36e229a; no voters:
I20260810 14:49:20.519091 13461 leader_election.cc:288] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:49:20.519209 13468 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:20.519829 13468 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [term 1 LEADER]: Becoming Leader. State: Replica: 1286f8c44fa94dd58cb6d29ac36e229a, State: Running, Role: LEADER
I20260810 14:49:20.520411 13468 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1286f8c44fa94dd58cb6d29ac36e229a" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44917 } }
I20260810 14:49:20.521322 13461 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:49:20.523591 13470 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1286f8c44fa94dd58cb6d29ac36e229a. Latest consensus state: current_term: 1 leader_uuid: "1286f8c44fa94dd58cb6d29ac36e229a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1286f8c44fa94dd58cb6d29ac36e229a" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44917 } } }
I20260810 14:49:20.524219 13470 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:20.523679 13469 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1286f8c44fa94dd58cb6d29ac36e229a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1286f8c44fa94dd58cb6d29ac36e229a" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44917 } } }
I20260810 14:49:20.524863 13469 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:20.526217 13476 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:49:20.531879 13476 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:49:20.541210 13476 catalog_manager.cc:1347] Generated new cluster ID: e5cb15fc65074bdb874d3e75f9abbaad
I20260810 14:49:20.541332 13476 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:49:20.565590 13476 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:49:20.567528 13476 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:49:20.577324 13476 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a: Generated new TSK 0
I20260810 14:49:20.578316 13476 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:49:20.655822 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:44917
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:21.002307 13492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:21.002579 13492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:21.002704 13492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:21.010361 13492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:21.010483 13492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:21.010639 13492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:49:21.021307 13492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:44917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:21.022799 13492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:21.024210 13492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:21.037892 13504 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:21.037954 13502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:21.038173 13500 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:21.040638 13492 server_base.cc:1034] running on GCE node
I20260810 14:49:21.041268 13492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:21.042217 13492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:21.043459 13492 hybrid_clock.cc:648] HybridClock initialized: now 1786373361043411 us; error 51 us; skew 500 ppm
I20260810 14:49:21.043780 13492 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:21 dist-test-slave-vkjp krb5kdc[13371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:21.047003 13492 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:49:21.048581 13492 webserver.cc:460] Webserver started at http://127.24.215.129:37983/ using document root <none> and password file <none>
I20260810 14:49:21.049268 13492 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:21.049365 13492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:21.049624 13492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:21.052486 13492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "20ddb0896a824fa78d07b4e5d5b53b65"
format_stamp: "Formatted at 2026-08-10 14:49:21 on dist-test-slave-vkjp"
server_key: "e11b0a4bd5d6904c491add2a946736c4"
server_key_iv: "e8f68a7d9c32a93099e0e8d9711b80e6"
server_key_version: "encryptionkey@0"
I20260810 14:49:21.053231 13492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "20ddb0896a824fa78d07b4e5d5b53b65"
format_stamp: "Formatted at 2026-08-10 14:49:21 on dist-test-slave-vkjp"
server_key: "e11b0a4bd5d6904c491add2a946736c4"
server_key_iv: "e8f68a7d9c32a93099e0e8d9711b80e6"
server_key_version: "encryptionkey@0"
I20260810 14:49:21.058313 13492 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:49:21.061750 13512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:21.063059 13492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:49:21.063297 13492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "20ddb0896a824fa78d07b4e5d5b53b65"
format_stamp: "Formatted at 2026-08-10 14:49:21 on dist-test-slave-vkjp"
server_key: "e11b0a4bd5d6904c491add2a946736c4"
server_key_iv: "e8f68a7d9c32a93099e0e8d9711b80e6"
server_key_version: "encryptionkey@0"
I20260810 14:49:21.063476 13492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:21.077323 13492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:21.081848 13492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:21.082086 13492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:21.082953 13492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:21.084268 13492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:21.084344 13492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:21.084429 13492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:21.084478 13492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:21.117993 13492 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:44007
I20260810 14:49:21.118040 13643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:44007 every 8 connection(s)
I20260810 14:49:21.119297 13492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:49:21.129034 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 13492
I20260810 14:49:21.129271 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:49:21.129969 25438 external_mini_cluster.cc:1428] Setting key cb312061fffcba666330f700be4d1cee
Aug 10 14:49:21 dist-test-slave-vkjp krb5kdc[13371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:49:21.139989 13646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:21.121918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:43099 (local address 127.24.215.190:44917)
0810 14:49:21.122395 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:21.122399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:21.123372 (+ 973us) server_negotiation.cc:407] Connection header received
0810 14:49:21.125218 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:21.125225 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:21.125311 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:21.125430 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:21.127070 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:21.127842 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:21.129504 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:21.129921 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:21.130064 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:21.134144 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:21.134150 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:21.134153 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:21.134183 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:21.136683 (+ 2500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:21.137599 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:21.137603 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:21.137605 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:21.137727 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:21.138206 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:21.138209 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:21.138211 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:21.138513 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:21.138623 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:21.139550 (+ 927us) server_negotiation.cc:299] Negotiation successful
0810 14:49:21.139712 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":252,"threads_started":1}
I20260810 14:49:21.141484 13647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:21.122331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:44917 (local address 127.24.215.129:43099)
0810 14:49:21.123213 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:21.123260 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:21.124941 (+ 1681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:21.125661 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:21.125669 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:21.126307 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:21.126884 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:21.126903 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:21.127992 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:21.128003 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:21.129358 (+ 1355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:21.129376 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:21.130150 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:21.130159 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:21.130391 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:21.131067 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:49:21.131127 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:21.133929 (+ 2802us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:21.136829 (+ 2900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:21.136839 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:21.136859 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:21.137486 (+ 627us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:21.137836 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:21.137843 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:21.137848 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:21.138097 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:21.138633 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:21.138644 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:21.139131 (+ 487us) client_negotiation.cc:769] Sending connection context
0810 14:49:21.139461 (+ 330us) client_negotiation.cc:240] Negotiation successful
0810 14:49:21.140028 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"thread_start_us":313,"threads_started":1}
I20260810 14:49:21.143023 13644 heartbeater.cc:347] Connected to a master server at 127.24.215.190:44917
I20260810 14:49:21.143343 13644 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:21.144134 13644 heartbeater.cc:511] Master 127.24.215.190:44917 requested a full tablet report, sending...
I20260810 14:49:21.146498 13422 ts_manager.cc:194] Registered new tserver with Master: 20ddb0896a824fa78d07b4e5d5b53b65 (127.24.215.129:44007)
I20260810 14:49:21.147982 13422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:43099
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:49:21.194167 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:44917
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:21.534286 13650 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:21.534658 13650 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:21.534819 13650 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:21.542299 13650 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:21.542402 13650 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:21.542596 13650 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:49:21.552232 13650 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:44917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:21.553568 13650 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:21.554848 13650 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:21.567732 13658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:21.567772 13661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:21.567838 13655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:21.568904 13650 server_base.cc:1034] running on GCE node
I20260810 14:49:21.569483 13650 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:21.570410 13650 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:21.571632 13650 hybrid_clock.cc:648] HybridClock initialized: now 1786373361571590 us; error 52 us; skew 500 ppm
I20260810 14:49:21.571939 13650 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:21 dist-test-slave-vkjp krb5kdc[13371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:21.575004 13650 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:49:21.576419 13650 webserver.cc:460] Webserver started at http://127.24.215.130:42457/ using document root <none> and password file <none>
I20260810 14:49:21.577103 13650 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:21.577250 13650 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:21.577567 13650 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:21.580462 13650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "b6dfbe201f3b48f380fe8168abc8e95c"
format_stamp: "Formatted at 2026-08-10 14:49:21 on dist-test-slave-vkjp"
server_key: "708f2a9b05577f95f80a0c2c8fd2a52b"
server_key_iv: "ae826b632f7db37fe0f9707d2df66371"
server_key_version: "encryptionkey@0"
I20260810 14:49:21.581236 13650 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "b6dfbe201f3b48f380fe8168abc8e95c"
format_stamp: "Formatted at 2026-08-10 14:49:21 on dist-test-slave-vkjp"
server_key: "708f2a9b05577f95f80a0c2c8fd2a52b"
server_key_iv: "ae826b632f7db37fe0f9707d2df66371"
server_key_version: "encryptionkey@0"
I20260810 14:49:21.586247 13650 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:49:21.591104 13668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:21.593168 13650 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260810 14:49:21.593329 13650 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "b6dfbe201f3b48f380fe8168abc8e95c"
format_stamp: "Formatted at 2026-08-10 14:49:21 on dist-test-slave-vkjp"
server_key: "708f2a9b05577f95f80a0c2c8fd2a52b"
server_key_iv: "ae826b632f7db37fe0f9707d2df66371"
server_key_version: "encryptionkey@0"
I20260810 14:49:21.593552 13650 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:21.618530 13650 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:21.622645 13650 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:21.622972 13650 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:21.623975 13650 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:21.625288 13650 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:21.625361 13650 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:21.625542 13650 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:21.625593 13650 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:21.659287 13650 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:39853
I20260810 14:49:21.659329 13795 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:39853 every 8 connection(s)
I20260810 14:49:21.660589 13650 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:49:21.671044 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 13650
I20260810 14:49:21.671275 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:49:21.671775 25438 external_mini_cluster.cc:1428] Setting key 5aa500b12f7d55bfd2202606a5f88f01
Aug 10 14:49:21 dist-test-slave-vkjp krb5kdc[13371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373361, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:49:21.681454 13798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:21.663147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:33755 (local address 127.24.215.190:44917)
0810 14:49:21.663658 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:21.663663 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:21.664455 (+ 792us) server_negotiation.cc:407] Connection header received
0810 14:49:21.666269 (+ 1814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:21.666274 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:21.666366 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:21.666491 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:21.668107 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:21.668839 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:21.670402 (+ 1563us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:21.671037 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:21.671196 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:21.675352 (+ 4156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:21.675358 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:21.675361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:21.675393 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:21.677992 (+ 2599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:21.679011 (+ 1019us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:21.679016 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:21.679018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:21.679206 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:21.679699 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:21.679703 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:21.679705 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:21.679955 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:21.680073 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:21.681030 (+ 957us) server_negotiation.cc:299] Negotiation successful
0810 14:49:21.681196 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":293,"threads_started":1}
I20260810 14:49:21.683008 13799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:21.663600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:44917 (local address 127.24.215.130:33755)
0810 14:49:21.664280 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:21.664321 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:21.666000 (+ 1679us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:21.666726 (+ 726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:21.666734 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:21.667309 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:21.667931 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:21.667949 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:21.669051 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:21.669061 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:21.670252 (+ 1191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:21.670264 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:21.671402 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:21.671417 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:21.671622 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:21.672392 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:49:21.672424 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:21.675135 (+ 2711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:21.678148 (+ 3013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:21.678159 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:21.678177 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:21.678880 (+ 703us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:21.679313 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:21.679320 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:21.679326 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:21.679587 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:21.680106 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:21.680117 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:21.680592 (+ 475us) client_negotiation.cc:769] Sending connection context
0810 14:49:21.680920 (+ 328us) client_negotiation.cc:240] Negotiation successful
0810 14:49:21.681541 (+ 621us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":261,"threads_started":1}
I20260810 14:49:21.684530 13796 heartbeater.cc:347] Connected to a master server at 127.24.215.190:44917
I20260810 14:49:21.684849 13796 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:21.685539 13796 heartbeater.cc:511] Master 127.24.215.190:44917 requested a full tablet report, sending...
I20260810 14:49:21.686964 13422 ts_manager.cc:194] Registered new tserver with Master: b6dfbe201f3b48f380fe8168abc8e95c (127.24.215.130:39853)
I20260810 14:49:21.687824 13422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:33755
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:49:21.738472 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:44917
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:40975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:22.080443 13802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:22.080700 13802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:22.080880 13802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:22.087936 13802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:22.088030 13802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:22.088223 13802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:49:22.097747 13802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:40975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:44917
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:22.099275 13802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:22.100627 13802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:22.113973 13809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:22.113925 13811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:22.114025 13814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:22.114879 13802 server_base.cc:1034] running on GCE node
I20260810 14:49:22.115526 13802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:22.116475 13802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:22.117745 13802 hybrid_clock.cc:648] HybridClock initialized: now 1786373362117701 us; error 46 us; skew 500 ppm
I20260810 14:49:22.118088 13802 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:22 dist-test-slave-vkjp krb5kdc[13371](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373362, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:22.122051 13802 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:49:22.124440 13802 webserver.cc:460] Webserver started at http://127.24.215.131:44695/ using document root <none> and password file <none>
I20260810 14:49:22.125113 13802 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:22.125258 13802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:22.125633 13802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:22.128579 13802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "c3020eaa75b14871a76e9f1ea33e9d8a"
format_stamp: "Formatted at 2026-08-10 14:49:22 on dist-test-slave-vkjp"
server_key: "da284d1061f5e4e2594f5ba055d2ed75"
server_key_iv: "19b1b44fb19bca37218ef69cfa568431"
server_key_version: "encryptionkey@0"
I20260810 14:49:22.129336 13802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "c3020eaa75b14871a76e9f1ea33e9d8a"
format_stamp: "Formatted at 2026-08-10 14:49:22 on dist-test-slave-vkjp"
server_key: "da284d1061f5e4e2594f5ba055d2ed75"
server_key_iv: "19b1b44fb19bca37218ef69cfa568431"
server_key_version: "encryptionkey@0"
I20260810 14:49:22.134593 13802 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 14:49:22.138515 13823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:22.139909 13802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 14:49:22.140093 13802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "c3020eaa75b14871a76e9f1ea33e9d8a"
format_stamp: "Formatted at 2026-08-10 14:49:22 on dist-test-slave-vkjp"
server_key: "da284d1061f5e4e2594f5ba055d2ed75"
server_key_iv: "19b1b44fb19bca37218ef69cfa568431"
server_key_version: "encryptionkey@0"
I20260810 14:49:22.140321 13802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:22.151492 13644 heartbeater.cc:503] Master 127.24.215.190:44917 was elected leader, sending a full tablet report...
I20260810 14:49:22.167743 13802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:22.171774 13802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:22.172071 13802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:22.173010 13802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:22.174322 13802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:22.174424 13802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:22.174571 13802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:22.174654 13802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:22.209928 13802 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:41635
I20260810 14:49:22.209973 13946 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:41635 every 8 connection(s)
I20260810 14:49:22.211323 13802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:49:22.216677 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 13802
I20260810 14:49:22.216881 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:49:22.217409 25438 external_mini_cluster.cc:1428] Setting key f002673a4bdfcec87365718a7ff8c75f
Aug 10 14:49:22 dist-test-slave-vkjp krb5kdc[13371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373362, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:49:22.231609 13949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:22.213860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:34583 (local address 127.24.215.190:44917)
0810 14:49:22.214357 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:22.214364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:22.215372 (+ 1008us) server_negotiation.cc:407] Connection header received
0810 14:49:22.217319 (+ 1947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:22.217325 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:22.217413 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:22.217548 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:22.219427 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.220408 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.222059 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.222551 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.222702 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:22.226093 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:22.226097 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:22.226101 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:22.226132 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:22.228830 (+ 2698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:22.229482 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:22.229486 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:22.229488 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:22.229608 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:22.229969 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:22.229972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:22.229974 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:22.230215 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:22.230326 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:22.230966 (+ 640us) server_negotiation.cc:299] Negotiation successful
0810 14:49:22.231214 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":220,"threads_started":1}
I20260810 14:49:22.232578 13951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:22.214449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:44917 (local address 127.24.215.131:34583)
0810 14:49:22.215208 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:22.215250 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:22.217057 (+ 1807us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:22.217776 (+ 719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:22.217784 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:22.218425 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:22.219237 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:22.219256 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.220570 (+ 1314us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.220577 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:22.221916 (+ 1339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:22.221933 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.222779 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.222784 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:22.222984 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:22.223717 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:49:22.223739 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:22.225912 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:22.228959 (+ 3047us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:22.228965 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:22.228976 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:22.229378 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:22.229696 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:22.229699 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:22.229702 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:22.229872 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:22.230313 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:22.230319 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:22.230606 (+ 287us) client_negotiation.cc:769] Sending connection context
0810 14:49:22.230812 (+ 206us) client_negotiation.cc:240] Negotiation successful
0810 14:49:22.231273 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":278,"threads_started":1}
I20260810 14:49:22.233987 13947 heartbeater.cc:347] Connected to a master server at 127.24.215.190:44917
I20260810 14:49:22.234251 13947 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:22.234889 13947 heartbeater.cc:511] Master 127.24.215.190:44917 requested a full tablet report, sending...
I20260810 14:49:22.236316 13422 ts_manager.cc:194] Registered new tserver with Master: c3020eaa75b14871a76e9f1ea33e9d8a (127.24.215.131:41635)
I20260810 14:49:22.237151 13422 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:34583
I20260810 14:49:22.245195 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:49:22.262252 13949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:22.249778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54750 (local address 127.24.215.190:44917)
0810 14:49:22.250010 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:22.250019 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:22.250503 (+ 484us) server_negotiation.cc:407] Connection header received
0810 14:49:22.250631 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:22.250634 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:22.250693 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:22.250770 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:22.252222 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.252924 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.254196 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.254585 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.254674 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:22.257288 (+ 2614us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:22.257293 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:22.257296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:22.257324 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:22.259957 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:22.260558 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:22.260561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:22.260564 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:22.260679 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:22.261015 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:22.261028 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:22.261030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:22.261317 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:22.261507 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:22.261777 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 14:49:22.261937 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 14:49:22.266280 13422 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54750:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:49:22.273200 13422 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:49:22.312170 13965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:22.285689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47266 (local address 127.24.215.131:41635)
0810 14:49:22.304049 (+ 18360us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:22.304060 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:22.304099 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:49:22.304202 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:22.304211 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:22.304511 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:22.304706 (+ 195us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:22.305483 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.306716 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.309559 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.310583 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.310770 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:22.310948 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:22.311623 (+ 675us) server_negotiation.cc:299] Negotiation successful
0810 14:49:22.311806 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18182,"thread_start_us":18010,"threads_started":1}
I20260810 14:49:22.312201 13966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:22.285262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:41635 (local address 127.0.0.1:47266)
0810 14:49:22.303153 (+ 17891us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:22.303191 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:22.303388 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:22.304735 (+ 1347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:22.304742 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:22.304774 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:22.305218 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:22.305232 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.306855 (+ 1623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.306869 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:22.309402 (+ 2533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:22.309415 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.311077 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.311083 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:22.311298 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:22.311350 (+ 52us) client_negotiation.cc:769] Sending connection context
0810 14:49:22.311550 (+ 200us) client_negotiation.cc:240] Negotiation successful
0810 14:49:22.311743 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17612,"mutex_wait_us":54,"thread_start_us":18443,"threads_started":1}
I20260810 14:49:22.313203 13963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:22.285289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:44007 (local address 127.0.0.1:46272)
0810 14:49:22.303993 (+ 18704us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:22.304015 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:22.304232 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:22.305174 (+ 942us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:22.305178 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:22.305208 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:22.305697 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:22.305710 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.307843 (+ 2133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.307859 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:22.310512 (+ 2653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:22.310524 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.312336 (+ 1812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.312345 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:22.312524 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:22.312556 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 14:49:22.312712 (+ 156us) client_negotiation.cc:240] Negotiation successful
0810 14:49:22.312847 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18571,"mutex_wait_us":23,"thread_start_us":18781,"threads_started":1}
I20260810 14:49:22.313299 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:22.286393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46272 (local address 127.24.215.129:44007)
0810 14:49:22.301878 (+ 15485us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:22.301891 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:22.304115 (+ 2224us) server_negotiation.cc:407] Connection header received
0810 14:49:22.304421 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:22.304429 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:22.304664 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:22.304877 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:22.305907 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.307693 (+ 1786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.310665 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.311799 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.312010 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:22.312175 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:22.312770 (+ 595us) server_negotiation.cc:299] Negotiation successful
0810 14:49:22.312959 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15265,"thread_start_us":15013,"threads_started":1}
I20260810 14:49:22.313457 13969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:22.286534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40082 (local address 127.24.215.130:39853)
0810 14:49:22.302348 (+ 15814us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:22.302372 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:22.304472 (+ 2100us) server_negotiation.cc:407] Connection header received
0810 14:49:22.304617 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:22.304627 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:22.304838 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:22.305016 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:22.306018 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.307642 (+ 1624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.310535 (+ 2893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.312016 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.312195 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:22.312358 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:22.312958 (+ 600us) server_negotiation.cc:299] Negotiation successful
0810 14:49:22.313131 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15636,"thread_start_us":15513,"threads_started":1}
I20260810 14:49:22.314232 13964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:22.285935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:39853 (local address 127.0.0.1:40082)
0810 14:49:22.304336 (+ 18401us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:22.304356 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:22.304481 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:22.305049 (+ 568us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:22.305053 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:22.305086 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:22.305733 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:22.305756 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.307826 (+ 2070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.307833 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:22.310339 (+ 2506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:22.310367 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.312525 (+ 2158us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.312532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:22.312736 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:22.312770 (+ 34us) client_negotiation.cc:769] Sending connection context
0810 14:49:22.313724 (+ 954us) client_negotiation.cc:240] Negotiation successful
0810 14:49:22.313878 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18270,"mutex_wait_us":38,"thread_start_us":16793,"threads_started":1}
I20260810 14:49:22.317034 13876 tablet_service.cc:1467] Processing CreateTablet for tablet 999611b4ea804fe3956f5ca6c0e5f0d6 (DEFAULT_TABLE table=test-table [id=15bdbead769142d8a4d1ad3b66f4a842]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:22.317034 13566 tablet_service.cc:1467] Processing CreateTablet for tablet 999611b4ea804fe3956f5ca6c0e5f0d6 (DEFAULT_TABLE table=test-table [id=15bdbead769142d8a4d1ad3b66f4a842]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:22.317437 13727 tablet_service.cc:1467] Processing CreateTablet for tablet 999611b4ea804fe3956f5ca6c0e5f0d6 (DEFAULT_TABLE table=test-table [id=15bdbead769142d8a4d1ad3b66f4a842]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:22.318586 13727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 999611b4ea804fe3956f5ca6c0e5f0d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:22.318591 13876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 999611b4ea804fe3956f5ca6c0e5f0d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:22.318593 13566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 999611b4ea804fe3956f5ca6c0e5f0d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:22.330179 13970 tablet_bootstrap.cc:492] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c: Bootstrap starting.
I20260810 14:49:22.333520 13971 tablet_bootstrap.cc:492] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65: Bootstrap starting.
I20260810 14:49:22.333702 13970 tablet_bootstrap.cc:654] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:22.335290 13973 tablet_bootstrap.cc:492] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a: Bootstrap starting.
I20260810 14:49:22.336660 13970 log.cc:824] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:22.337231 13971 tablet_bootstrap.cc:654] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:22.338449 13971 log.cc:824] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:22.338761 13973 tablet_bootstrap.cc:654] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:22.340061 13973 log.cc:824] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:22.340096 13970 tablet_bootstrap.cc:492] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c: No bootstrap required, opened a new log
I20260810 14:49:22.340395 13970 ts_tablet_manager.cc:1397] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260810 14:49:22.341650 13971 tablet_bootstrap.cc:492] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65: No bootstrap required, opened a new log
I20260810 14:49:22.341933 13971 ts_tablet_manager.cc:1397] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.005s
I20260810 14:49:22.342578 13973 tablet_bootstrap.cc:492] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a: No bootstrap required, opened a new log
I20260810 14:49:22.342832 13973 ts_tablet_manager.cc:1397] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260810 14:49:22.350950 13973 raft_consensus.cc:348] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3020eaa75b14871a76e9f1ea33e9d8a" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41635 } } peers { permanent_uuid: "20ddb0896a824fa78d07b4e5d5b53b65" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44007 } } peers { permanent_uuid: "b6dfbe201f3b48f380fe8168abc8e95c" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 39853 } }
I20260810 14:49:22.351390 13973 raft_consensus.cc:374] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:22.351498 13973 raft_consensus.cc:729] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3020eaa75b14871a76e9f1ea33e9d8a, State: Initialized, Role: FOLLOWER
I20260810 14:49:22.351792 13970 raft_consensus.cc:348] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3020eaa75b14871a76e9f1ea33e9d8a" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41635 } } peers { permanent_uuid: "20ddb0896a824fa78d07b4e5d5b53b65" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44007 } } peers { permanent_uuid: "b6dfbe201f3b48f380fe8168abc8e95c" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 39853 } }
I20260810 14:49:22.352214 13970 raft_consensus.cc:374] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:22.352321 13970 raft_consensus.cc:729] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6dfbe201f3b48f380fe8168abc8e95c, State: Initialized, Role: FOLLOWER
I20260810 14:49:22.352272 13973 consensus_queue.cc:260] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3020eaa75b14871a76e9f1ea33e9d8a" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41635 } } peers { permanent_uuid: "20ddb0896a824fa78d07b4e5d5b53b65" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44007 } } peers { permanent_uuid: "b6dfbe201f3b48f380fe8168abc8e95c" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 39853 } }
I20260810 14:49:22.352469 13971 raft_consensus.cc:348] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3020eaa75b14871a76e9f1ea33e9d8a" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41635 } } peers { permanent_uuid: "20ddb0896a824fa78d07b4e5d5b53b65" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44007 } } peers { permanent_uuid: "b6dfbe201f3b48f380fe8168abc8e95c" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 39853 } }
I20260810 14:49:22.352890 13971 raft_consensus.cc:374] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:22.352994 13971 raft_consensus.cc:729] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20ddb0896a824fa78d07b4e5d5b53b65, State: Initialized, Role: FOLLOWER
I20260810 14:49:22.353034 13970 consensus_queue.cc:260] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3020eaa75b14871a76e9f1ea33e9d8a" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41635 } } peers { permanent_uuid: "20ddb0896a824fa78d07b4e5d5b53b65" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44007 } } peers { permanent_uuid: "b6dfbe201f3b48f380fe8168abc8e95c" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 39853 } }
I20260810 14:49:22.353698 13947 heartbeater.cc:503] Master 127.24.215.190:44917 was elected leader, sending a full tablet report...
I20260810 14:49:22.353693 13971 consensus_queue.cc:260] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3020eaa75b14871a76e9f1ea33e9d8a" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41635 } } peers { permanent_uuid: "20ddb0896a824fa78d07b4e5d5b53b65" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44007 } } peers { permanent_uuid: "b6dfbe201f3b48f380fe8168abc8e95c" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 39853 } }
I20260810 14:49:22.355962 13971 ts_tablet_manager.cc:1428] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65: Time spent starting tablet: real 0.014s user 0.012s sys 0.003s
I20260810 14:49:22.356993 13796 heartbeater.cc:503] Master 127.24.215.190:44917 was elected leader, sending a full tablet report...
I20260810 14:49:22.357679 13970 ts_tablet_manager.cc:1428] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c: Time spent starting tablet: real 0.017s user 0.009s sys 0.008s
I20260810 14:49:22.357769 13973 ts_tablet_manager.cc:1428] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a: Time spent starting tablet: real 0.015s user 0.008s sys 0.004s
W20260810 14:49:22.371593 13645 tablet.cc:2369] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:49:22.412624 13797 tablet.cc:2369] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:49:22.463027 13948 tablet.cc:2369] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:49:22.711249 13978 raft_consensus.cc:482] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:49:22.711601 13978 raft_consensus.cc:504] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3020eaa75b14871a76e9f1ea33e9d8a" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41635 } } peers { permanent_uuid: "20ddb0896a824fa78d07b4e5d5b53b65" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44007 } } peers { permanent_uuid: "b6dfbe201f3b48f380fe8168abc8e95c" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 39853 } }
I20260810 14:49:22.713423 13978 leader_election.cc:288] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c3020eaa75b14871a76e9f1ea33e9d8a (127.24.215.131:41635), 20ddb0896a824fa78d07b4e5d5b53b65 (127.24.215.129:44007)
I20260810 14:49:22.720409 13965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:22.713932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:48369 (local address 127.24.215.131:41635)
0810 14:49:22.714148 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:22.714154 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:22.714902 (+ 748us) server_negotiation.cc:407] Connection header received
0810 14:49:22.715167 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:22.715171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:22.715230 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:22.715327 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:22.715886 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.716605 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.718584 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.719443 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.719635 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:22.719716 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:22.720000 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 14:49:22.720131 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260810 14:49:22.720460 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:22.714066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:39227 (local address 127.24.215.129:44007)
0810 14:49:22.714305 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:22.714314 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:22.715052 (+ 738us) server_negotiation.cc:407] Connection header received
0810 14:49:22.715280 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:22.715283 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:22.715343 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:22.715444 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:22.716093 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.716810 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.718595 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.719454 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.719635 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:22.719727 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:22.720054 (+ 327us) server_negotiation.cc:299] Negotiation successful
0810 14:49:22.720188 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:49:22.720578 13980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:22.713883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:41635 (local address 127.24.215.130:48369)
0810 14:49:22.714779 (+ 896us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:22.714814 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:22.714986 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:22.715347 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:22.715351 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:22.715375 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:22.715738 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:22.715746 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.716743 (+ 997us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.716748 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:22.718426 (+ 1678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:22.718438 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.719680 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.719686 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:22.719841 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:22.719864 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 14:49:22.719973 (+ 109us) client_negotiation.cc:240] Negotiation successful
0810 14:49:22.720124 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":743,"mutex_wait_us":2,"thread_start_us":433,"threads_started":1}
I20260810 14:49:22.720974 13981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:22.713932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:44007 (local address 127.24.215.130:39227)
0810 14:49:22.714938 (+ 1006us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:22.714953 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:22.715133 (+ 180us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:22.715483 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:22.715487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:22.715529 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:22.715925 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:22.715934 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.716952 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.716959 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:22.718434 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:22.718447 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.719737 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.719746 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:22.719890 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:22.719913 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 14:49:22.720016 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 14:49:22.720138 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":899,"mutex_wait_us":24,"thread_start_us":686,"threads_started":1}
I20260810 14:49:22.721726 13898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "999611b4ea804fe3956f5ca6c0e5f0d6" candidate_uuid: "b6dfbe201f3b48f380fe8168abc8e95c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3020eaa75b14871a76e9f1ea33e9d8a" is_pre_election: true
I20260810 14:49:22.722000 13589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "999611b4ea804fe3956f5ca6c0e5f0d6" candidate_uuid: "b6dfbe201f3b48f380fe8168abc8e95c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ddb0896a824fa78d07b4e5d5b53b65" is_pre_election: true
I20260810 14:49:22.722344 13898 raft_consensus.cc:2454] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6dfbe201f3b48f380fe8168abc8e95c in term 0.
I20260810 14:49:22.722564 13589 raft_consensus.cc:2454] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6dfbe201f3b48f380fe8168abc8e95c in term 0.
I20260810 14:49:22.723039 13673 leader_election.cc:302] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6dfbe201f3b48f380fe8168abc8e95c, c3020eaa75b14871a76e9f1ea33e9d8a; no voters:
I20260810 14:49:22.723522 13978 raft_consensus.cc:2781] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:49:22.723698 13978 raft_consensus.cc:482] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:49:22.723805 13978 raft_consensus.cc:3037] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:22.726084 13978 raft_consensus.cc:504] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3020eaa75b14871a76e9f1ea33e9d8a" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41635 } } peers { permanent_uuid: "20ddb0896a824fa78d07b4e5d5b53b65" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44007 } } peers { permanent_uuid: "b6dfbe201f3b48f380fe8168abc8e95c" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 39853 } }
I20260810 14:49:22.726759 13978 leader_election.cc:288] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [CANDIDATE]: Term 1 election: Requested vote from peers c3020eaa75b14871a76e9f1ea33e9d8a (127.24.215.131:41635), 20ddb0896a824fa78d07b4e5d5b53b65 (127.24.215.129:44007)
I20260810 14:49:22.727190 13898 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "999611b4ea804fe3956f5ca6c0e5f0d6" candidate_uuid: "b6dfbe201f3b48f380fe8168abc8e95c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c3020eaa75b14871a76e9f1ea33e9d8a"
I20260810 14:49:22.727327 13589 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "999611b4ea804fe3956f5ca6c0e5f0d6" candidate_uuid: "b6dfbe201f3b48f380fe8168abc8e95c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "20ddb0896a824fa78d07b4e5d5b53b65"
I20260810 14:49:22.727443 13898 raft_consensus.cc:3037] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:22.727559 13589 raft_consensus.cc:3037] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:22.729637 13898 raft_consensus.cc:2454] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6dfbe201f3b48f380fe8168abc8e95c in term 1.
I20260810 14:49:22.729828 13589 raft_consensus.cc:2454] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6dfbe201f3b48f380fe8168abc8e95c in term 1.
I20260810 14:49:22.730036 13673 leader_election.cc:302] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b6dfbe201f3b48f380fe8168abc8e95c, c3020eaa75b14871a76e9f1ea33e9d8a; no voters:
I20260810 14:49:22.730345 13978 raft_consensus.cc:2781] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:22.730822 13978 raft_consensus.cc:686] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [term 1 LEADER]: Becoming Leader. State: Replica: b6dfbe201f3b48f380fe8168abc8e95c, State: Running, Role: LEADER
I20260810 14:49:22.731390 13978 consensus_queue.cc:237] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3020eaa75b14871a76e9f1ea33e9d8a" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41635 } } peers { permanent_uuid: "20ddb0896a824fa78d07b4e5d5b53b65" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44007 } } peers { permanent_uuid: "b6dfbe201f3b48f380fe8168abc8e95c" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 39853 } }
I20260810 14:49:22.735939 13422 catalog_manager.cc:5495] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c reported cstate change: term changed from 0 to 1, leader changed from <none> to b6dfbe201f3b48f380fe8168abc8e95c (127.24.215.130). New cstate: current_term: 1 leader_uuid: "b6dfbe201f3b48f380fe8168abc8e95c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c3020eaa75b14871a76e9f1ea33e9d8a" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41635 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "20ddb0896a824fa78d07b4e5d5b53b65" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44007 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6dfbe201f3b48f380fe8168abc8e95c" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 39853 } health_report { overall_health: HEALTHY } } }
I20260810 14:49:22.861799 13986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:22.855733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40090 (local address 127.24.215.130:39853)
0810 14:49:22.856437 (+ 704us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:22.856443 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:22.856474 (+ 31us) server_negotiation.cc:407] Connection header received
0810 14:49:22.856594 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:22.856603 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:22.856733 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:22.856868 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:49:22.857267 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.858013 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.859705 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.860135 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.860222 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:22.860671 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:49:22.861201 (+ 530us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:49:22.861309 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:22.861428 (+ 119us) server_negotiation.cc:299] Negotiation successful
0810 14:49:22.861492 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":429,"threads_started":1}
I20260810 14:49:22.869639 13589 raft_consensus.cc:1261] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65 [term 1 FOLLOWER]: Refusing update from remote peer b6dfbe201f3b48f380fe8168abc8e95c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:49:22.869638 13898 raft_consensus.cc:1261] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a [term 1 FOLLOWER]: Refusing update from remote peer b6dfbe201f3b48f380fe8168abc8e95c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:49:22.870694 13982 consensus_queue.cc:1035] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [LEADER]: Connected to new peer: Peer: permanent_uuid: "20ddb0896a824fa78d07b4e5d5b53b65" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 44007 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:49:22.871047 13978 consensus_queue.cc:1035] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [LEADER]: Connected to new peer: Peer: permanent_uuid: "c3020eaa75b14871a76e9f1ea33e9d8a" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 41635 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:49:22.881502 13987 mvcc.cc:205] Tried to move back new op lower bound from 7316985294303772672 to 7316985293751939072. Current Snapshot: MvccSnapshot[applied={T|T < 7316985294303772672}]
I20260810 14:49:22.888515 13990 mvcc.cc:205] Tried to move back new op lower bound from 7316985294303772672 to 7316985293751939072. Current Snapshot: MvccSnapshot[applied={T|T < 7316985294303772672}]
I20260810 14:49:22.897738 13422 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:49:22.898059 13422 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54750:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:49:22.901273 13422 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1286f8c44fa94dd58cb6d29ac36e229a: Sending DeleteTablet for 3 replicas of tablet 999611b4ea804fe3956f5ca6c0e5f0d6
I20260810 14:49:22.902375 13876 tablet_service.cc:1514] Processing DeleteTablet for tablet 999611b4ea804fe3956f5ca6c0e5f0d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:22 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:47266
I20260810 14:49:22.902439 13566 tablet_service.cc:1514] Processing DeleteTablet for tablet 999611b4ea804fe3956f5ca6c0e5f0d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:22 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:46272
I20260810 14:49:22.902695 13727 tablet_service.cc:1514] Processing DeleteTablet for tablet 999611b4ea804fe3956f5ca6c0e5f0d6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:22 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:40082
I20260810 14:49:22.903515 13998 tablet_replica.cc:330] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a: stopping tablet replica
I20260810 14:49:22.904034 13998 raft_consensus.cc:2229] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:49:22.904634 13998 raft_consensus.cc:2258] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:22.904780 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 13492
I20260810 14:49:22.906908 13998 ts_tablet_manager.cc:1905] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:22.915563 13999 tablet_replica.cc:330] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65: stopping tablet replica
I20260810 14:49:22.916162 13999 raft_consensus.cc:2229] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:49:22.916795 13999 raft_consensus.cc:2258] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:22.917692 13998 ts_tablet_manager.cc:1918] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:49:22.917721 14002 tablet_replica.cc:330] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c: stopping tablet replica
I20260810 14:49:22.917913 13998 log.cc:1196] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/999611b4ea804fe3956f5ca6c0e5f0d6
I20260810 14:49:22.918112 14002 raft_consensus.cc:2229] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:49:22.918702 13998 ts_tablet_manager.cc:1939] T 999611b4ea804fe3956f5ca6c0e5f0d6 P c3020eaa75b14871a76e9f1ea33e9d8a: Deleting consensus metadata
I20260810 14:49:22.918817 14002 raft_consensus.cc:2258] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:22.920883 14002 ts_tablet_manager.cc:1905] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:22.920954 13406 catalog_manager.cc:4841] TS c3020eaa75b14871a76e9f1ea33e9d8a (127.24.215.131:41635): tablet 999611b4ea804fe3956f5ca6c0e5f0d6 (table test-table [id=15bdbead769142d8a4d1ad3b66f4a842]) successfully deleted
I20260810 14:49:22.922972 13999 ts_tablet_manager.cc:1905] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 10 14:49:22 dist-test-slave-vkjp krb5kdc[13371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:49:22.929239 14002 ts_tablet_manager.cc:1918] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:49:22.929452 14002 log.cc:1196] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/999611b4ea804fe3956f5ca6c0e5f0d6
I20260810 14:49:22.930056 14002 ts_tablet_manager.cc:1939] T 999611b4ea804fe3956f5ca6c0e5f0d6 P b6dfbe201f3b48f380fe8168abc8e95c: Deleting consensus metadata
I20260810 14:49:22.931182 13999 ts_tablet_manager.cc:1918] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:49:22.931442 13999 log.cc:1196] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/999611b4ea804fe3956f5ca6c0e5f0d6
I20260810 14:49:22.932013 13403 catalog_manager.cc:4841] TS b6dfbe201f3b48f380fe8168abc8e95c (127.24.215.130:39853): tablet 999611b4ea804fe3956f5ca6c0e5f0d6 (table test-table [id=15bdbead769142d8a4d1ad3b66f4a842]) successfully deleted
I20260810 14:49:22.932093 13999 ts_tablet_manager.cc:1939] T 999611b4ea804fe3956f5ca6c0e5f0d6 P 20ddb0896a824fa78d07b4e5d5b53b65: Deleting consensus metadata
I20260810 14:49:22.934203 13404 catalog_manager.cc:4841] TS 20ddb0896a824fa78d07b4e5d5b53b65 (127.24.215.129:44007): tablet 999611b4ea804fe3956f5ca6c0e5f0d6 (table test-table [id=15bdbead769142d8a4d1ad3b66f4a842]) successfully deleted
I20260810 14:49:22.938055 13968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:22.906289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46282 (local address 127.24.215.129:44007)
0810 14:49:22.906468 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:22.906474 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:22.917986 (+ 11512us) server_negotiation.cc:407] Connection header received
0810 14:49:22.918204 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:22.918212 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:22.918305 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:22.918429 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:22.920364 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.921598 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.923569 (+ 1971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:22.924285 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:22.924391 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:22.930373 (+ 5982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:22.930390 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:22.930409 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:22.930454 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:22.934610 (+ 4156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:22.935644 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:22.935652 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:22.935655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:22.935798 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:22.936250 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:22.936254 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:22.936256 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:22.936738 (+ 482us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:22.936959 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:22.937467 (+ 508us) server_negotiation.cc:299] Negotiation successful
0810 14:49:22.937710 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":65}
I20260810 14:49:22.938751 13638 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:23.029305 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 13492
I20260810 14:49:23.053088 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 13650
Aug 10 14:49:23 dist-test-slave-vkjp krb5kdc[13371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:49:23.072247 13986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:23.054065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40092 (local address 127.24.215.130:39853)
0810 14:49:23.054240 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:23.054246 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:23.054264 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:49:23.054410 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:23.054414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:23.054484 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:23.054577 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:23.056320 (+ 1743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:23.057435 (+ 1115us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:23.059753 (+ 2318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:23.060238 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:23.060436 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:23.065861 (+ 5425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:23.065878 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:23.065896 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:23.065950 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:23.069240 (+ 3290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:23.069998 (+ 758us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:23.070005 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:23.070008 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:23.070139 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:23.070548 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:23.070552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:23.070554 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:23.070961 (+ 407us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:23.071294 (+ 333us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:23.071618 (+ 324us) server_negotiation.cc:299] Negotiation successful
0810 14:49:23.071879 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260810 14:49:23.073089 13791 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:23.186908 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 13650
I20260810 14:49:23.210321 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 13802
Aug 10 14:49:23 dist-test-slave-vkjp krb5kdc[13371](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373359, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:49:23.232187 13965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:23.211254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47280 (local address 127.24.215.131:41635)
0810 14:49:23.211467 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:23.211479 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:23.211503 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:49:23.211731 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:23.211735 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:23.211809 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:23.211923 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:23.213194 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:23.214008 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:23.215615 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:23.216044 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:23.216206 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:23.226269 (+ 10063us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:23.226288 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:23.226302 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:23.226341 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:23.229398 (+ 3057us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:23.230098 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:23.230105 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:23.230107 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:23.230230 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:23.230639 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:23.230643 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:23.230645 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:23.230993 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:23.231321 (+ 328us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:23.231631 (+ 310us) server_negotiation.cc:299] Negotiation successful
0810 14:49:23.231859 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 14:49:23.232846 13942 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:23.321626 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 13802
I20260810 14:49:23.343581 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 13385
I20260810 14:49:23.344805 13455 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:23.456786 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 13385
2026-08-10T14:49:23Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (4949 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-10T14:49:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:49:23Z Disabled control of system clock
I20260810 14:49:23.551409 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:46599
--webserver_interface=127.24.215.190
--webserver_port=0
--builtin_ntp_servers=127.24.215.148:36443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:46599
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:49:23.892055 14011 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:23.892329 14011 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:23.892411 14011 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:23.899505 14011 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:49:23.899606 14011 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:23.899652 14011 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:23.899708 14011 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:49:23.899752 14011 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:49:23.908780 14011 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:36443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:46599
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:46599
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:23.910028 14011 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:23.911827 14011 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:23.923842 14018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:23.923751 14021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:23.923807 14019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:23.924717 14011 server_base.cc:1034] running on GCE node
I20260810 14:49:23.925443 14011 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:23.926801 14011 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:23.928028 14011 hybrid_clock.cc:648] HybridClock initialized: now 1786373363927978 us; error 48 us; skew 500 ppm
I20260810 14:49:23.928380 14011 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:49:23.930639 14011 webserver.cc:460] Webserver started at http://127.24.215.190:44195/ using document root <none> and password file <none>
I20260810 14:49:23.931412 14011 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:23.931566 14011 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:23.931880 14011 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:23.934731 14011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "05e5039da8d4458ab99d2bad7e0ab74d"
format_stamp: "Formatted at 2026-08-10 14:49:23 on dist-test-slave-vkjp"
server_key: "bf66d38a756e54a71e021869ddc5bd84"
server_key_iv: "90809c45aa12d6d7deefb93d7d7c6fc4"
server_key_version: "encryptionkey@0"
I20260810 14:49:23.935576 14011 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "05e5039da8d4458ab99d2bad7e0ab74d"
format_stamp: "Formatted at 2026-08-10 14:49:23 on dist-test-slave-vkjp"
server_key: "bf66d38a756e54a71e021869ddc5bd84"
server_key_iv: "90809c45aa12d6d7deefb93d7d7c6fc4"
server_key_version: "encryptionkey@0"
I20260810 14:49:23.940804 14011 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 14:49:23.944204 14029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:23.945421 14011 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:49:23.945608 14011 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "05e5039da8d4458ab99d2bad7e0ab74d"
format_stamp: "Formatted at 2026-08-10 14:49:23 on dist-test-slave-vkjp"
server_key: "bf66d38a756e54a71e021869ddc5bd84"
server_key_iv: "90809c45aa12d6d7deefb93d7d7c6fc4"
server_key_version: "encryptionkey@0"
I20260810 14:49:23.945819 14011 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:23.967224 14011 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:23.968235 14011 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:23.968568 14011 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:23.989235 14011 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:46599
I20260810 14:49:23.989255 14084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:46599 every 8 connection(s)
I20260810 14:49:23.990638 14011 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:49:23.994360 14085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:23.998319 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 14011
I20260810 14:49:23.998529 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:49:23.999116 25438 external_mini_cluster.cc:1428] Setting key 954cf9a05f447e8d34283243f7ef97ae
I20260810 14:49:24.007308 14085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d: Bootstrap starting.
I20260810 14:49:24.010625 14085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:24.011854 14085 log.cc:824] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:24.013073 14088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:24.001093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59230 (local address 127.24.215.190:46599)
0810 14:49:24.002064 (+ 971us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:24.002075 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:24.002114 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:49:24.003837 (+ 1723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:24.003862 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:24.004344 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:24.004737 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:24.005352 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:24.006623 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:24.008207 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:24.008927 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:24.009139 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:24.010382 (+ 1243us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:24.010390 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:24.010403 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:24.010971 (+ 568us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:24.011122 (+ 151us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:24.011164 (+ 42us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:24.011287 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:24.011511 (+ 224us) server_negotiation.cc:299] Negotiation successful
0810 14:49:24.011808 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":366,"threads_started":1}
I20260810 14:49:24.015066 14085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d: No bootstrap required, opened a new log
I20260810 14:49:24.022825 14085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e5039da8d4458ab99d2bad7e0ab74d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46599 } }
I20260810 14:49:24.023244 14085 raft_consensus.cc:374] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:24.023370 14085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 05e5039da8d4458ab99d2bad7e0ab74d, State: Initialized, Role: FOLLOWER
I20260810 14:49:24.023912 14085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e5039da8d4458ab99d2bad7e0ab74d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46599 } }
I20260810 14:49:24.024152 14085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:49:24.024288 14085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:49:24.024426 14085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:24.026893 14085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e5039da8d4458ab99d2bad7e0ab74d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46599 } }
I20260810 14:49:24.027436 14085 leader_election.cc:302] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 05e5039da8d4458ab99d2bad7e0ab74d; no voters:
I20260810 14:49:24.028092 14085 leader_election.cc:288] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:49:24.028241 14091 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:24.029630 14091 raft_consensus.cc:686] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [term 1 LEADER]: Becoming Leader. State: Replica: 05e5039da8d4458ab99d2bad7e0ab74d, State: Running, Role: LEADER
I20260810 14:49:24.030121 14091 consensus_queue.cc:237] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e5039da8d4458ab99d2bad7e0ab74d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46599 } }
I20260810 14:49:24.030369 14085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:49:24.033174 14093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 05e5039da8d4458ab99d2bad7e0ab74d. Latest consensus state: current_term: 1 leader_uuid: "05e5039da8d4458ab99d2bad7e0ab74d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e5039da8d4458ab99d2bad7e0ab74d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46599 } } }
I20260810 14:49:24.033294 14092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "05e5039da8d4458ab99d2bad7e0ab74d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "05e5039da8d4458ab99d2bad7e0ab74d" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 46599 } } }
I20260810 14:49:24.033691 14093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:24.033772 14092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:24.036782 14099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:49:24.042526 14099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:49:24.051712 14099 catalog_manager.cc:1347] Generated new cluster ID: 853c489ebeff4a5883c6da84178c5c0f
I20260810 14:49:24.051815 14099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:49:24.068109 14099 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:49:24.069314 14099 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:49:24.087393 14099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d: Generated new TSK 0
I20260810 14:49:24.088243 14099 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:49:24.102193 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46599
--builtin_ntp_servers=127.24.215.148:36443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:49:24.439286 14112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:24.439674 14112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:24.439824 14112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:24.447041 14112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:24.447176 14112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:24.447369 14112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:49:24.456621 14112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:36443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:24.457886 14112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:24.459666 14112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:24.473150 14117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:24.473209 14118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:24.473237 14121 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:24.474289 14112 server_base.cc:1034] running on GCE node
I20260810 14:49:24.474869 14112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:24.475797 14112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:24.477021 14112 hybrid_clock.cc:648] HybridClock initialized: now 1786373364476978 us; error 37 us; skew 500 ppm
I20260810 14:49:24.477340 14112 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:49:24.479516 14112 webserver.cc:460] Webserver started at http://127.24.215.129:34079/ using document root <none> and password file <none>
I20260810 14:49:24.480185 14112 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:24.480310 14112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:24.480633 14112 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:24.483494 14112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "991f62d9555d4c168c26436cd9ff8534"
format_stamp: "Formatted at 2026-08-10 14:49:24 on dist-test-slave-vkjp"
server_key: "b9f7572162c5060111325ce716066cec"
server_key_iv: "92660f27e4f760345b08c3eb4b59eb64"
server_key_version: "encryptionkey@0"
I20260810 14:49:24.484279 14112 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "991f62d9555d4c168c26436cd9ff8534"
format_stamp: "Formatted at 2026-08-10 14:49:24 on dist-test-slave-vkjp"
server_key: "b9f7572162c5060111325ce716066cec"
server_key_iv: "92660f27e4f760345b08c3eb4b59eb64"
server_key_version: "encryptionkey@0"
I20260810 14:49:24.489233 14112 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260810 14:49:24.492610 14127 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:24.493923 14112 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 14:49:24.494098 14112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "991f62d9555d4c168c26436cd9ff8534"
format_stamp: "Formatted at 2026-08-10 14:49:24 on dist-test-slave-vkjp"
server_key: "b9f7572162c5060111325ce716066cec"
server_key_iv: "92660f27e4f760345b08c3eb4b59eb64"
server_key_version: "encryptionkey@0"
I20260810 14:49:24.494311 14112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:24.505844 14112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:24.506595 14112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:24.506891 14112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:24.507866 14112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:24.509315 14112 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:24.509433 14112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:24.509564 14112 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:24.509650 14112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:24.543553 14112 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:43045
I20260810 14:49:24.543601 14255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:43045 every 8 connection(s)
I20260810 14:49:24.544889 14112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:49:24.552613 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 14112
I20260810 14:49:24.552793 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:49:24.553431 25438 external_mini_cluster.cc:1428] Setting key 93dd7d0b48ef2c2b3b1876cd3c2c46c6
I20260810 14:49:24.556244 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46599
--builtin_ntp_servers=127.24.215.148:36443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:49:24.558027 14258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:24.547467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:55287 (local address 127.24.215.190:46599)
0810 14:49:24.547919 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:24.547924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:24.549057 (+ 1133us) server_negotiation.cc:407] Connection header received
0810 14:49:24.550904 (+ 1847us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:24.550908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:24.550974 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:24.551070 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:24.552106 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:24.552945 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:24.554985 (+ 2040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:24.555591 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:24.555733 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:24.557164 (+ 1431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:24.557170 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:24.557175 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:24.557201 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:24.557244 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:24.557256 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:24.557351 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:24.557578 (+ 227us) server_negotiation.cc:299] Negotiation successful
0810 14:49:24.557681 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":248,"threads_started":1}
I20260810 14:49:24.559031 14259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:24.547969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:46599 (local address 127.24.215.129:55287)
0810 14:49:24.548854 (+ 885us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:24.548916 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:24.550668 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:24.551319 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:24.551326 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:24.551420 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:24.551961 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:24.551978 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:24.553097 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:24.553104 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:24.554830 (+ 1726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:24.554842 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:24.555828 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:24.555836 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:24.556093 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:24.556839 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:49:24.556842 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:24.556896 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:49:24.556981 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:24.557353 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:24.557368 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:24.557380 (+ 12us) client_negotiation.cc:769] Sending connection context
0810 14:49:24.557571 (+ 191us) client_negotiation.cc:240] Negotiation successful
0810 14:49:24.557979 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":469,"thread_start_us":270,"threads_started":1}
I20260810 14:49:24.560848 14256 heartbeater.cc:347] Connected to a master server at 127.24.215.190:46599
I20260810 14:49:24.561193 14256 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:24.561962 14256 heartbeater.cc:511] Master 127.24.215.190:46599 requested a full tablet report, sending...
I20260810 14:49:24.564316 14047 ts_manager.cc:194] Registered new tserver with Master: 991f62d9555d4c168c26436cd9ff8534 (127.24.215.129:43045)
I20260810 14:49:24.566511 14047 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.215.129:55287
W20260810 14:49:24.892849 14260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:24.893174 14260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:24.893334 14260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:24.900635 14260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:24.900763 14260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:24.900939 14260 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:49:24.909991 14260 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:36443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:24.911306 14260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:24.913162 14260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:24.926520 14265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:24.926481 14268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:24.926437 14266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:24.927352 14260 server_base.cc:1034] running on GCE node
I20260810 14:49:24.927968 14260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:24.928912 14260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:24.930102 14260 hybrid_clock.cc:648] HybridClock initialized: now 1786373364930058 us; error 53 us; skew 500 ppm
I20260810 14:49:24.930446 14260 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:49:24.932802 14260 webserver.cc:460] Webserver started at http://127.24.215.130:32965/ using document root <none> and password file <none>
I20260810 14:49:24.933501 14260 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:24.933636 14260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:24.933957 14260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:24.937019 14260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "5983c4f959434ce283d295205fcbaebf"
format_stamp: "Formatted at 2026-08-10 14:49:24 on dist-test-slave-vkjp"
server_key: "43ba809096615ee51f4af8777bc2d6a3"
server_key_iv: "9c2ad40bed080bd8bfb33175f5c16046"
server_key_version: "encryptionkey@0"
I20260810 14:49:24.937871 14260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "5983c4f959434ce283d295205fcbaebf"
format_stamp: "Formatted at 2026-08-10 14:49:24 on dist-test-slave-vkjp"
server_key: "43ba809096615ee51f4af8777bc2d6a3"
server_key_iv: "9c2ad40bed080bd8bfb33175f5c16046"
server_key_version: "encryptionkey@0"
I20260810 14:49:24.943189 14260 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260810 14:49:24.946521 14274 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:24.947773 14260 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 14:49:24.947948 14260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "5983c4f959434ce283d295205fcbaebf"
format_stamp: "Formatted at 2026-08-10 14:49:24 on dist-test-slave-vkjp"
server_key: "43ba809096615ee51f4af8777bc2d6a3"
server_key_iv: "9c2ad40bed080bd8bfb33175f5c16046"
server_key_version: "encryptionkey@0"
I20260810 14:49:24.948160 14260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:24.981178 14260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:24.982076 14260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:24.982399 14260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:24.983475 14260 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:24.984970 14260 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:24.985131 14260 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:24.985258 14260 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:24.985354 14260 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:25.022562 14260 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:35697
I20260810 14:49:25.022652 14398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:35697 every 8 connection(s)
I20260810 14:49:25.023905 14260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:49:25.026188 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 14260
I20260810 14:49:25.026422 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:49:25.026947 25438 external_mini_cluster.cc:1428] Setting key 6990aababc4b74cf3560d25d51e8fc89
I20260810 14:49:25.030244 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46599
--builtin_ntp_servers=127.24.215.148:36443
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:49:25.038240 14258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.026480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:39679 (local address 127.24.215.190:46599)
0810 14:49:25.026695 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:25.026701 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:25.028270 (+ 1569us) server_negotiation.cc:407] Connection header received
0810 14:49:25.030698 (+ 2428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:25.030703 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:25.030790 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:25.030880 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:25.032300 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.033002 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.035420 (+ 2418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.035821 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.035913 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.037540 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:25.037545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:25.037548 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:25.037566 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:25.037597 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:25.037608 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:25.037672 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:25.037941 (+ 269us) server_negotiation.cc:299] Negotiation successful
0810 14:49:25.038017 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260810 14:49:25.039718 14403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.027020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:46599 (local address 127.24.215.130:39679)
0810 14:49:25.028092 (+ 1072us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:25.028154 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:25.030393 (+ 2239us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:25.031166 (+ 773us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:25.031180 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:25.031296 (+ 116us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:25.032132 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.032160 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.033155 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.033162 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.035283 (+ 2121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.035305 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.036040 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.036049 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.036401 (+ 352us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.037191 (+ 790us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:49:25.037197 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:25.037248 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:49:25.037349 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:25.037700 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:25.037713 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:25.037724 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 14:49:25.037913 (+ 189us) client_negotiation.cc:240] Negotiation successful
0810 14:49:25.038319 (+ 406us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"thread_start_us":307,"threads_started":1}
I20260810 14:49:25.041328 14401 heartbeater.cc:347] Connected to a master server at 127.24.215.190:46599
I20260810 14:49:25.041643 14401 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:25.042568 14401 heartbeater.cc:511] Master 127.24.215.190:46599 requested a full tablet report, sending...
I20260810 14:49:25.044072 14047 ts_manager.cc:194] Registered new tserver with Master: 5983c4f959434ce283d295205fcbaebf (127.24.215.130:35697)
I20260810 14:49:25.044893 14047 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.215.130:39679
W20260810 14:49:25.372741 14404 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:25.373070 14404 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:25.373229 14404 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:25.380506 14404 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:25.380635 14404 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:25.380805 14404 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:49:25.389900 14404 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:36443
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:46599
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:25.391297 14404 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:25.393047 14404 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:25.407229 14410 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:25.407148 14414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:25.407107 14411 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:25.408473 14404 server_base.cc:1034] running on GCE node
I20260810 14:49:25.409255 14404 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:25.410320 14404 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:25.411562 14404 hybrid_clock.cc:648] HybridClock initialized: now 1786373365411508 us; error 47 us; skew 500 ppm
I20260810 14:49:25.411904 14404 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:49:25.414287 14404 webserver.cc:460] Webserver started at http://127.24.215.131:41881/ using document root <none> and password file <none>
I20260810 14:49:25.415035 14404 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:25.415199 14404 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:25.415561 14404 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:25.418483 14404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38"
format_stamp: "Formatted at 2026-08-10 14:49:25 on dist-test-slave-vkjp"
server_key: "16020d9aa0f9bd3776025d8de741c0ae"
server_key_iv: "947b90f24bd1c2acd5240d403027d812"
server_key_version: "encryptionkey@0"
I20260810 14:49:25.419392 14404 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38"
format_stamp: "Formatted at 2026-08-10 14:49:25 on dist-test-slave-vkjp"
server_key: "16020d9aa0f9bd3776025d8de741c0ae"
server_key_iv: "947b90f24bd1c2acd5240d403027d812"
server_key_version: "encryptionkey@0"
I20260810 14:49:25.424659 14404 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 14:49:25.428259 14422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:25.429551 14404 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260810 14:49:25.429734 14404 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38"
format_stamp: "Formatted at 2026-08-10 14:49:25 on dist-test-slave-vkjp"
server_key: "16020d9aa0f9bd3776025d8de741c0ae"
server_key_iv: "947b90f24bd1c2acd5240d403027d812"
server_key_version: "encryptionkey@0"
I20260810 14:49:25.429970 14404 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:25.464452 14404 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:25.465349 14404 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:25.465687 14404 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:25.466702 14404 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:25.468291 14404 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:25.468418 14404 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:25.468556 14404 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:25.468647 14404 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:25.504060 14404 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:34775
I20260810 14:49:25.504123 14552 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:34775 every 8 connection(s)
I20260810 14:49:25.505466 14404 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:49:25.512223 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 14404
I20260810 14:49:25.512440 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:49:25.513025 25438 external_mini_cluster.cc:1428] Setting key 3c2827b08ad3971d5c2877a7cd6bea84
I20260810 14:49:25.518586 14258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.508044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:38507 (local address 127.24.215.190:46599)
0810 14:49:25.508273 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:25.508279 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:25.509652 (+ 1373us) server_negotiation.cc:407] Connection header received
0810 14:49:25.511467 (+ 1815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:25.511471 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:25.511537 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:25.511621 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:25.513043 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.514014 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.516171 (+ 2157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.516570 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.516664 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.517924 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:25.517929 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:25.517932 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:25.517948 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:25.517979 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:25.517998 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:25.518066 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:25.518256 (+ 190us) server_negotiation.cc:299] Negotiation successful
0810 14:49:25.518329 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 14:49:25.519690 14555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.508605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:46599 (local address 127.24.215.131:38507)
0810 14:49:25.509500 (+ 895us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:25.509542 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:25.511225 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:25.511875 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:25.511887 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:25.512006 (+ 119us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:25.512863 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.512892 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.514180 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.514184 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.515997 (+ 1813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.516032 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.516784 (+ 752us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.516789 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.516999 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.517634 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:49:25.517637 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:25.517675 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:49:25.517755 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:25.518079 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:25.518087 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:25.518093 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:49:25.518228 (+ 135us) client_negotiation.cc:240] Negotiation successful
0810 14:49:25.518517 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":274,"threads_started":1}
I20260810 14:49:25.520897 14553 heartbeater.cc:347] Connected to a master server at 127.24.215.190:46599
I20260810 14:49:25.521236 14553 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:25.521996 14553 heartbeater.cc:511] Master 127.24.215.190:46599 requested a full tablet report, sending...
I20260810 14:49:25.523597 14047 ts_manager.cc:194] Registered new tserver with Master: fb3fed59d94d43c7b48dcc4f17dc1c38 (127.24.215.131:34775)
I20260810 14:49:25.524405 14047 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.215.131:38507
I20260810 14:49:25.528321 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:49:25.538300 14258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.532456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59234 (local address 127.24.215.190:46599)
0810 14:49:25.532686 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:25.532695 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:25.533023 (+ 328us) server_negotiation.cc:407] Connection header received
0810 14:49:25.533211 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:25.533215 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:25.533285 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:25.533363 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:25.533898 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.534664 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.536043 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.536506 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.536611 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.537625 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:25.537629 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:25.537632 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:25.537648 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:25.537679 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:25.537690 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:25.537758 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:25.537898 (+ 140us) server_negotiation.cc:299] Negotiation successful
0810 14:49:25.537967 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 14:49:25.542048 14047 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:59234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:49:25.548995 14047 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:49:25.569657 14256 heartbeater.cc:503] Master 127.24.215.190:46599 was elected leader, sending a full tablet report...
I20260810 14:49:25.574416 14567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.561859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:35697 (local address 127.0.0.1:56990)
0810 14:49:25.562877 (+ 1018us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:25.562933 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:25.563144 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:25.564727 (+ 1583us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:25.564739 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:25.564783 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:25.565360 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.565387 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.568024 (+ 2637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.568033 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.570926 (+ 2893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.570948 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.573474 (+ 2526us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.573485 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.573716 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.573780 (+ 64us) client_negotiation.cc:769] Sending connection context
0810 14:49:25.573887 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 14:49:25.574095 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"thread_start_us":547,"threads_started":1}
I20260810 14:49:25.575615 14568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.562128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56990 (local address 127.24.215.130:35697)
0810 14:49:25.563910 (+ 1782us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:25.563923 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:25.563961 (+ 38us) server_negotiation.cc:407] Connection header received
0810 14:49:25.564095 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:25.564106 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:25.564432 (+ 326us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:25.565642 (+ 1210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:25.565877 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.567825 (+ 1948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.571279 (+ 3454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.573203 (+ 1924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.574931 (+ 1728us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.575157 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:25.575223 (+ 66us) server_negotiation.cc:299] Negotiation successful
0810 14:49:25.575345 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1600,"thread_start_us":1490,"threads_started":1}
I20260810 14:49:25.584409 14570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.562816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51874 (local address 127.24.215.129:43045)
0810 14:49:25.576277 (+ 13461us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:25.576285 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:25.576310 (+ 25us) server_negotiation.cc:407] Connection header received
0810 14:49:25.576418 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:25.576427 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:25.576688 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:25.576891 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:25.577597 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.579289 (+ 1692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.582011 (+ 2722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.583335 (+ 1324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.583542 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.583765 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:25.583899 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 14:49:25.584075 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13314,"thread_start_us":13205,"threads_started":1}
I20260810 14:49:25.584964 14569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.562607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:43045 (local address 127.0.0.1:51874)
0810 14:49:25.573223 (+ 10616us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:25.573248 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:25.573401 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:25.576882 (+ 3481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:25.576886 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:25.576924 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:25.577390 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.577402 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.579460 (+ 2058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.579465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.581779 (+ 2314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.581797 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.583544 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.583550 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.583717 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.583749 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 14:49:25.583851 (+ 102us) client_negotiation.cc:240] Negotiation successful
0810 14:49:25.583987 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10433,"mutex_wait_us":25,"thread_start_us":10268,"threads_started":1}
I20260810 14:49:25.585276 14567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.563323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:34775 (local address 127.0.0.1:56948)
0810 14:49:25.574657 (+ 11334us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:25.574674 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:25.574763 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:25.578254 (+ 3491us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:25.578259 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:25.578286 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:25.578522 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.578532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.580785 (+ 2253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.580793 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.582567 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.582582 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.584413 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.584419 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.584606 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.584639 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:49:25.584705 (+ 66us) client_negotiation.cc:240] Negotiation successful
0810 14:49:25.584845 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11249,"mutex_wait_us":5,"thread_start_us":17538,"threads_started":1}
I20260810 14:49:25.585246 14332 tablet_service.cc:1467] Processing CreateTablet for tablet 785ebcc72d35493889cdfb96967238fa (DEFAULT_TABLE table=test-table [id=010f2b28f22f4ce980442785e6afd992]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:25.586548 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.563921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56948 (local address 127.24.215.131:34775)
0810 14:49:25.577671 (+ 13750us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:25.577683 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:25.577716 (+ 33us) server_negotiation.cc:407] Connection header received
0810 14:49:25.577814 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:25.577821 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:25.578047 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:25.578682 (+ 635us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:25.578950 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.580602 (+ 1652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.582740 (+ 2138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.584210 (+ 1470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.585751 (+ 1541us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.585984 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:25.586047 (+ 63us) server_negotiation.cc:299] Negotiation successful
0810 14:49:25.586202 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13560,"thread_start_us":13381,"threads_started":1}
I20260810 14:49:25.587393 14332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 785ebcc72d35493889cdfb96967238fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:25.588932 14182 tablet_service.cc:1467] Processing CreateTablet for tablet 785ebcc72d35493889cdfb96967238fa (DEFAULT_TABLE table=test-table [id=010f2b28f22f4ce980442785e6afd992]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:25.590466 14182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 785ebcc72d35493889cdfb96967238fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:25.592482 14474 tablet_service.cc:1467] Processing CreateTablet for tablet 785ebcc72d35493889cdfb96967238fa (DEFAULT_TABLE table=test-table [id=010f2b28f22f4ce980442785e6afd992]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:25.594684 14474 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 785ebcc72d35493889cdfb96967238fa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:25.604408 14573 tablet_bootstrap.cc:492] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf: Bootstrap starting.
I20260810 14:49:25.606493 14574 tablet_bootstrap.cc:492] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534: Bootstrap starting.
I20260810 14:49:25.608479 14573 tablet_bootstrap.cc:654] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:25.609519 14574 tablet_bootstrap.cc:654] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:25.610059 14573 log.cc:824] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:25.610262 14575 tablet_bootstrap.cc:492] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38: Bootstrap starting.
I20260810 14:49:25.610864 14574 log.cc:824] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:25.615206 14575 tablet_bootstrap.cc:654] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:25.616436 14575 log.cc:824] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:25.617184 14574 tablet_bootstrap.cc:492] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534: No bootstrap required, opened a new log
I20260810 14:49:25.617401 14573 tablet_bootstrap.cc:492] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf: No bootstrap required, opened a new log
I20260810 14:49:25.617492 14574 ts_tablet_manager.cc:1397] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.000s
I20260810 14:49:25.617707 14573 ts_tablet_manager.cc:1397] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf: Time spent bootstrapping tablet: real 0.014s user 0.006s sys 0.005s
I20260810 14:49:25.619580 14575 tablet_bootstrap.cc:492] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38: No bootstrap required, opened a new log
I20260810 14:49:25.619834 14575 ts_tablet_manager.cc:1397] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260810 14:49:25.624900 14574 raft_consensus.cc:348] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4f959434ce283d295205fcbaebf" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35697 } } peers { permanent_uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34775 } } peers { permanent_uuid: "991f62d9555d4c168c26436cd9ff8534" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 43045 } }
I20260810 14:49:25.625443 14574 raft_consensus.cc:374] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:25.625581 14574 raft_consensus.cc:729] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 991f62d9555d4c168c26436cd9ff8534, State: Initialized, Role: FOLLOWER
I20260810 14:49:25.626377 14574 consensus_queue.cc:260] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4f959434ce283d295205fcbaebf" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35697 } } peers { permanent_uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34775 } } peers { permanent_uuid: "991f62d9555d4c168c26436cd9ff8534" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 43045 } }
I20260810 14:49:25.628322 14574 ts_tablet_manager.cc:1428] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534: Time spent starting tablet: real 0.011s user 0.011s sys 0.000s
I20260810 14:49:25.628542 14573 raft_consensus.cc:348] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4f959434ce283d295205fcbaebf" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35697 } } peers { permanent_uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34775 } } peers { permanent_uuid: "991f62d9555d4c168c26436cd9ff8534" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 43045 } }
I20260810 14:49:25.629081 14573 raft_consensus.cc:374] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:25.629216 14573 raft_consensus.cc:729] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5983c4f959434ce283d295205fcbaebf, State: Initialized, Role: FOLLOWER
I20260810 14:49:25.629901 14573 consensus_queue.cc:260] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4f959434ce283d295205fcbaebf" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35697 } } peers { permanent_uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34775 } } peers { permanent_uuid: "991f62d9555d4c168c26436cd9ff8534" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 43045 } }
I20260810 14:49:25.630551 14575 raft_consensus.cc:348] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4f959434ce283d295205fcbaebf" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35697 } } peers { permanent_uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34775 } } peers { permanent_uuid: "991f62d9555d4c168c26436cd9ff8534" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 43045 } }
I20260810 14:49:25.631124 14575 raft_consensus.cc:374] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:25.631259 14575 raft_consensus.cc:729] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb3fed59d94d43c7b48dcc4f17dc1c38, State: Initialized, Role: FOLLOWER
I20260810 14:49:25.632158 14575 consensus_queue.cc:260] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4f959434ce283d295205fcbaebf" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35697 } } peers { permanent_uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34775 } } peers { permanent_uuid: "991f62d9555d4c168c26436cd9ff8534" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 43045 } }
I20260810 14:49:25.635838 14553 heartbeater.cc:503] Master 127.24.215.190:46599 was elected leader, sending a full tablet report...
I20260810 14:49:25.636786 14575 ts_tablet_manager.cc:1428] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38: Time spent starting tablet: real 0.017s user 0.010s sys 0.007s
I20260810 14:49:25.636946 14573 ts_tablet_manager.cc:1428] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf: Time spent starting tablet: real 0.019s user 0.020s sys 0.000s
I20260810 14:49:25.637317 14401 heartbeater.cc:503] Master 127.24.215.190:46599 was elected leader, sending a full tablet report...
W20260810 14:49:25.757210 14554 tablet.cc:2369] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:49:25.775946 14402 tablet.cc:2369] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:49:25.797451 14257 tablet.cc:2369] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:49:25.919982 14580 raft_consensus.cc:482] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:49:25.920322 14580 raft_consensus.cc:504] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4f959434ce283d295205fcbaebf" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35697 } } peers { permanent_uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34775 } } peers { permanent_uuid: "991f62d9555d4c168c26436cd9ff8534" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 43045 } }
I20260810 14:49:25.922361 14580 leader_election.cc:288] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb3fed59d94d43c7b48dcc4f17dc1c38 (127.24.215.131:34775), 991f62d9555d4c168c26436cd9ff8534 (127.24.215.129:43045)
I20260810 14:49:25.923538 14579 raft_consensus.cc:482] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:49:25.923863 14579 raft_consensus.cc:504] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4f959434ce283d295205fcbaebf" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35697 } } peers { permanent_uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34775 } } peers { permanent_uuid: "991f62d9555d4c168c26436cd9ff8534" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 43045 } }
I20260810 14:49:25.926005 14579 leader_election.cc:288] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5983c4f959434ce283d295205fcbaebf (127.24.215.130:35697), fb3fed59d94d43c7b48dcc4f17dc1c38 (127.24.215.131:34775)
I20260810 14:49:25.933719 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.922786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:43045 (local address 127.24.215.130:38553)
0810 14:49:25.923817 (+ 1031us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:25.923836 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:25.924050 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:25.924583 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:25.924589 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:25.924647 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:25.925243 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.925261 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.926558 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.926562 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.929197 (+ 2635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.929221 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.931854 (+ 2633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.931863 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.932107 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.932157 (+ 50us) client_negotiation.cc:769] Sending connection context
0810 14:49:25.932972 (+ 815us) client_negotiation.cc:240] Negotiation successful
0810 14:49:25.933154 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":895,"mutex_wait_us":51,"spinlock_wait_cycles":21888,"thread_start_us":755,"threads_started":1}
I20260810 14:49:25.933777 14570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.923063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:38553 (local address 127.24.215.129:43045)
0810 14:49:25.923358 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:25.923366 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:25.923936 (+ 570us) server_negotiation.cc:407] Connection header received
0810 14:49:25.924293 (+ 357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:25.924299 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:25.924404 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:25.924518 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:25.925420 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.926352 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.929537 (+ 3185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.931320 (+ 1783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.931500 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.931608 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:25.932348 (+ 740us) server_negotiation.cc:299] Negotiation successful
0810 14:49:25.932552 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134}
I20260810 14:49:25.933903 14572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.922810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:52115 (local address 127.24.215.131:34775)
0810 14:49:25.923057 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:25.923065 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:25.923847 (+ 782us) server_negotiation.cc:407] Connection header received
0810 14:49:25.924188 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:25.924196 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:25.924306 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:25.924400 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:25.925244 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.926348 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.931407 (+ 5059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.932833 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.932988 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.933070 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:25.933482 (+ 412us) server_negotiation.cc:299] Negotiation successful
0810 14:49:25.933613 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260810 14:49:25.935245 14582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.922753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:34775 (local address 127.24.215.130:52115)
0810 14:49:25.923715 (+ 962us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:25.923738 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:25.923988 (+ 250us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:25.924433 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:25.924437 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:25.924470 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:25.925055 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.925073 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.926525 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.926532 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.931148 (+ 4616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.931166 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.933120 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.933129 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.933280 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.933303 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 14:49:25.933421 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 14:49:25.933560 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"mutex_wait_us":40,"spinlock_wait_cycles":768,"thread_start_us":531,"threads_started":1}
I20260810 14:49:25.935316 14204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "785ebcc72d35493889cdfb96967238fa" candidate_uuid: "5983c4f959434ce283d295205fcbaebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991f62d9555d4c168c26436cd9ff8534" is_pre_election: true
I20260810 14:49:25.935896 14204 raft_consensus.cc:2454] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5983c4f959434ce283d295205fcbaebf in term 0.
I20260810 14:49:25.936619 14496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "785ebcc72d35493889cdfb96967238fa" candidate_uuid: "5983c4f959434ce283d295205fcbaebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38" is_pre_election: true
I20260810 14:49:25.936852 14279 leader_election.cc:302] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5983c4f959434ce283d295205fcbaebf, 991f62d9555d4c168c26436cd9ff8534; no voters:
I20260810 14:49:25.937278 14496 raft_consensus.cc:2454] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5983c4f959434ce283d295205fcbaebf in term 0.
I20260810 14:49:25.937335 14580 raft_consensus.cc:2781] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:49:25.937472 14584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.926311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:35697 (local address 127.24.215.129:57577)
0810 14:49:25.927188 (+ 877us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:25.927211 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:25.927423 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:25.927978 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:25.927982 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:25.928013 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:25.928519 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.928535 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.930242 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.930247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.932855 (+ 2608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.932871 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.936149 (+ 3278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.936156 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.936411 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.936456 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 14:49:25.936929 (+ 473us) client_negotiation.cc:240] Negotiation successful
0810 14:49:25.937074 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":710,"thread_start_us":615,"threads_started":1}
I20260810 14:49:25.937495 14580 raft_consensus.cc:482] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:49:25.937601 14580 raft_consensus.cc:3037] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:25.938442 14568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.926502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:57577 (local address 127.24.215.130:35697)
0810 14:49:25.926785 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:25.926794 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:25.927309 (+ 515us) server_negotiation.cc:407] Connection header received
0810 14:49:25.927598 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:25.927603 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:25.927707 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:25.927782 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:25.928715 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.930014 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.934359 (+ 4345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.935734 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.935882 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.935957 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:25.936637 (+ 680us) server_negotiation.cc:299] Negotiation successful
0810 14:49:25.936787 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:49:25.939471 14352 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "785ebcc72d35493889cdfb96967238fa" candidate_uuid: "991f62d9555d4c168c26436cd9ff8534" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5983c4f959434ce283d295205fcbaebf" is_pre_election: true
I20260810 14:49:25.940740 14580 raft_consensus.cc:504] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4f959434ce283d295205fcbaebf" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35697 } } peers { permanent_uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34775 } } peers { permanent_uuid: "991f62d9555d4c168c26436cd9ff8534" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 43045 } }
I20260810 14:49:25.941344 14352 raft_consensus.cc:2379] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 991f62d9555d4c168c26436cd9ff8534 in current term 1: Already voted for candidate 5983c4f959434ce283d295205fcbaebf in this term.
I20260810 14:49:25.941579 14580 leader_election.cc:288] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [CANDIDATE]: Term 1 election: Requested vote from peers fb3fed59d94d43c7b48dcc4f17dc1c38 (127.24.215.131:34775), 991f62d9555d4c168c26436cd9ff8534 (127.24.215.129:43045)
I20260810 14:49:25.941763 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.927380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:34775 (local address 127.24.215.129:43189)
0810 14:49:25.934560 (+ 7180us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:25.934585 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:25.934725 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:25.935343 (+ 618us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:25.935348 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:25.935372 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:25.935778 (+ 406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.935789 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.937346 (+ 1557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.937354 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.939440 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:25.939457 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.940932 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.940942 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:25.941166 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.941201 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 14:49:25.941292 (+ 91us) client_negotiation.cc:240] Negotiation successful
0810 14:49:25.941424 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7036,"thread_start_us":6814,"threads_started":1}
I20260810 14:49:25.941922 14586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:25.927942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:43189 (local address 127.24.215.131:34775)
0810 14:49:25.928756 (+ 814us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:25.928764 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:25.934697 (+ 5933us) server_negotiation.cc:407] Connection header received
0810 14:49:25.934856 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:25.934862 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:25.934947 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:25.935045 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:25.936010 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.937150 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.939670 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:25.940720 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:25.940864 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:25.940913 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:25.941377 (+ 464us) server_negotiation.cc:299] Negotiation successful
0810 14:49:25.941533 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":515,"threads_started":1}
I20260810 14:49:25.942188 14204 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "785ebcc72d35493889cdfb96967238fa" candidate_uuid: "5983c4f959434ce283d295205fcbaebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "991f62d9555d4c168c26436cd9ff8534"
I20260810 14:49:25.942378 14204 raft_consensus.cc:3037] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:25.942658 14496 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "785ebcc72d35493889cdfb96967238fa" candidate_uuid: "991f62d9555d4c168c26436cd9ff8534" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38" is_pre_election: true
I20260810 14:49:25.942891 14496 raft_consensus.cc:2454] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 991f62d9555d4c168c26436cd9ff8534 in term 0.
I20260810 14:49:25.943552 14128 leader_election.cc:302] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 991f62d9555d4c168c26436cd9ff8534, fb3fed59d94d43c7b48dcc4f17dc1c38; no voters: 5983c4f959434ce283d295205fcbaebf
I20260810 14:49:25.945096 14204 raft_consensus.cc:2454] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5983c4f959434ce283d295205fcbaebf in term 1.
I20260810 14:49:25.945283 14579 raft_consensus.cc:2741] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260810 14:49:25.945659 14279 leader_election.cc:302] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5983c4f959434ce283d295205fcbaebf, 991f62d9555d4c168c26436cd9ff8534; no voters:
I20260810 14:49:25.945581 14495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "785ebcc72d35493889cdfb96967238fa" candidate_uuid: "5983c4f959434ce283d295205fcbaebf" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38"
I20260810 14:49:25.945858 14495 raft_consensus.cc:3037] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:25.946146 14580 raft_consensus.cc:2781] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:25.946691 14580 raft_consensus.cc:686] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [term 1 LEADER]: Becoming Leader. State: Replica: 5983c4f959434ce283d295205fcbaebf, State: Running, Role: LEADER
I20260810 14:49:25.947245 14580 consensus_queue.cc:237] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4f959434ce283d295205fcbaebf" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35697 } } peers { permanent_uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34775 } } peers { permanent_uuid: "991f62d9555d4c168c26436cd9ff8534" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 43045 } }
I20260810 14:49:25.948421 14495 raft_consensus.cc:2454] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5983c4f959434ce283d295205fcbaebf in term 1.
I20260810 14:49:25.951568 14047 catalog_manager.cc:5495] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf reported cstate change: term changed from 0 to 1, leader changed from <none> to 5983c4f959434ce283d295205fcbaebf (127.24.215.130). New cstate: current_term: 1 leader_uuid: "5983c4f959434ce283d295205fcbaebf" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5983c4f959434ce283d295205fcbaebf" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 35697 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34775 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "991f62d9555d4c168c26436cd9ff8534" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 43045 } health_report { overall_health: UNKNOWN } } }
I20260810 14:49:26.030030 14568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:26.023599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57000 (local address 127.24.215.130:35697)
0810 14:49:26.023828 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:26.023834 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:26.023958 (+ 124us) server_negotiation.cc:407] Connection header received
0810 14:49:26.024203 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:26.024209 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:26.024321 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:26.024486 (+ 165us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:49:26.024805 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:26.026026 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:26.027518 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:26.028242 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:26.028366 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:26.028821 (+ 455us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:49:26.029398 (+ 577us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:49:26.029538 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:26.029641 (+ 103us) server_negotiation.cc:299] Negotiation successful
0810 14:49:26.029718 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 14:49:26.039398 14495 raft_consensus.cc:1261] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38 [term 1 FOLLOWER]: Refusing update from remote peer 5983c4f959434ce283d295205fcbaebf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:49:26.039394 14204 raft_consensus.cc:1261] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534 [term 1 FOLLOWER]: Refusing update from remote peer 5983c4f959434ce283d295205fcbaebf: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:49:26.040304 14587 consensus_queue.cc:1035] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb3fed59d94d43c7b48dcc4f17dc1c38" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:49:26.040684 14580 consensus_queue.cc:1035] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [LEADER]: Connected to new peer: Peer: permanent_uuid: "991f62d9555d4c168c26436cd9ff8534" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 43045 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:49:26.051813 14590 mvcc.cc:205] Tried to move back new op lower bound from 7316985307286376448 to 7316985306923601920. Current Snapshot: MvccSnapshot[applied={T|T < 7316985307286376448}]
I20260810 14:49:26.056950 14592 mvcc.cc:205] Tried to move back new op lower bound from 7316985307286376448 to 7316985306923601920. Current Snapshot: MvccSnapshot[applied={T|T < 7316985307286376448}]
I20260810 14:49:26.065974 14046 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:59234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:49:26.066484 14046 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:59234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:49:26.070056 14046 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 05e5039da8d4458ab99d2bad7e0ab74d: Sending DeleteTablet for 3 replicas of tablet 785ebcc72d35493889cdfb96967238fa
I20260810 14:49:26.071269 14332 tablet_service.cc:1514] Processing DeleteTablet for tablet 785ebcc72d35493889cdfb96967238fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:26 UTC) from {username='slave'} at 127.0.0.1:56990
I20260810 14:49:26.071502 14182 tablet_service.cc:1514] Processing DeleteTablet for tablet 785ebcc72d35493889cdfb96967238fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:26 UTC) from {username='slave'} at 127.0.0.1:51874
I20260810 14:49:26.071451 14474 tablet_service.cc:1514] Processing DeleteTablet for tablet 785ebcc72d35493889cdfb96967238fa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:26 UTC) from {username='slave'} at 127.0.0.1:56948
I20260810 14:49:26.073802 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 14112
I20260810 14:49:26.083180 14600 tablet_replica.cc:330] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534: stopping tablet replica
I20260810 14:49:26.083791 14600 raft_consensus.cc:2229] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:49:26.084499 14600 raft_consensus.cc:2258] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:26.087577 14600 ts_tablet_manager.cc:1905] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:26.095728 14602 tablet_replica.cc:330] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38: stopping tablet replica
I20260810 14:49:26.096304 14602 raft_consensus.cc:2229] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:49:26.097265 14602 raft_consensus.cc:2258] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:26.097771 14599 tablet_replica.cc:330] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf: stopping tablet replica
I20260810 14:49:26.098117 14600 ts_tablet_manager.cc:1918] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:49:26.098186 14599 raft_consensus.cc:2229] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:49:26.098471 14600 log.cc:1196] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/785ebcc72d35493889cdfb96967238fa
I20260810 14:49:26.099251 14600 ts_tablet_manager.cc:1939] T 785ebcc72d35493889cdfb96967238fa P 991f62d9555d4c168c26436cd9ff8534: Deleting consensus metadata
I20260810 14:49:26.099740 14602 ts_tablet_manager.cc:1905] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:26.101353 14033 catalog_manager.cc:4841] TS 991f62d9555d4c168c26436cd9ff8534 (127.24.215.129:43045): tablet 785ebcc72d35493889cdfb96967238fa (table test-table [id=010f2b28f22f4ce980442785e6afd992]) successfully deleted
I20260810 14:49:26.102312 14599 raft_consensus.cc:2258] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:26.104892 14570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:26.075258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51888 (local address 127.24.215.129:43045)
0810 14:49:26.075443 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:26.075450 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:26.095999 (+ 20549us) server_negotiation.cc:407] Connection header received
0810 14:49:26.096199 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:26.096205 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:26.096319 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:26.096433 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:26.097618 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:26.098791 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:26.100889 (+ 2098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:26.101537 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:26.101640 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:26.103134 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:26.103145 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:26.103162 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:26.103946 (+ 784us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:26.104009 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:26.104034 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:26.104127 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:26.104381 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 14:49:26.104513 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 14:49:26.105629 14599 ts_tablet_manager.cc:1905] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:26.106041 14249 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:26.106899 14602 ts_tablet_manager.cc:1918] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:49:26.107252 14602 log.cc:1196] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/785ebcc72d35493889cdfb96967238fa
I20260810 14:49:26.108052 14602 ts_tablet_manager.cc:1939] T 785ebcc72d35493889cdfb96967238fa P fb3fed59d94d43c7b48dcc4f17dc1c38: Deleting consensus metadata
I20260810 14:49:26.110396 14030 catalog_manager.cc:4841] TS fb3fed59d94d43c7b48dcc4f17dc1c38 (127.24.215.131:34775): tablet 785ebcc72d35493889cdfb96967238fa (table test-table [id=010f2b28f22f4ce980442785e6afd992]) successfully deleted
I20260810 14:49:26.117025 14599 ts_tablet_manager.cc:1918] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:49:26.117213 14599 log.cc:1196] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/785ebcc72d35493889cdfb96967238fa
I20260810 14:49:26.117794 14599 ts_tablet_manager.cc:1939] T 785ebcc72d35493889cdfb96967238fa P 5983c4f959434ce283d295205fcbaebf: Deleting consensus metadata
I20260810 14:49:26.119750 14030 catalog_manager.cc:4841] TS 5983c4f959434ce283d295205fcbaebf (127.24.215.130:35697): tablet 785ebcc72d35493889cdfb96967238fa (table test-table [id=010f2b28f22f4ce980442785e6afd992]) successfully deleted
I20260810 14:49:26.204416 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 14112
I20260810 14:49:26.233556 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 14260
I20260810 14:49:26.241896 14568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:26.234436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57006 (local address 127.24.215.130:35697)
0810 14:49:26.234642 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:26.234648 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:26.234682 (+ 34us) server_negotiation.cc:407] Connection header received
0810 14:49:26.234789 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:26.234795 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:26.234924 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:26.235035 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:26.235478 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:26.236636 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:26.238071 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:26.238757 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:26.238887 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:26.240150 (+ 1263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:26.240162 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:26.240182 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:26.241048 (+ 866us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:26.241117 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:26.241149 (+ 32us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:26.241258 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:26.241408 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 14:49:26.241516 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 14:49:26.242717 14394 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:26.355139 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 14260
I20260810 14:49:26.378038 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 14404
I20260810 14:49:26.384922 14586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:26.378862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56956 (local address 127.24.215.131:34775)
0810 14:49:26.379116 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:26.379124 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:26.379148 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:49:26.379234 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:26.379237 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:26.379322 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:26.379406 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:26.379902 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:26.380576 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:26.381934 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:26.382353 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:26.382438 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:26.383658 (+ 1220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:26.383668 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:26.383683 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:26.384250 (+ 567us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:26.384297 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:26.384317 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:26.384401 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:26.384531 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 14:49:26.384609 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260810 14:49:26.385608 14548 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:26.474661 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 14404
I20260810 14:49:26.495994 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 14011
I20260810 14:49:26.497071 14080 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:26.606176 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 14011
2026-08-10T14:49:26Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (3141 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:49:26 dist-test-slave-vkjp krb5kdc[14613](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:49:26 dist-test-slave-vkjp krb5kdc[14613](info): set up 2 sockets
Aug 10 14:49:26 dist-test-slave-vkjp krb5kdc[14613](info): commencing operation
krb5kdc: starting...
W20260810 14:49:27.820955 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.003s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:49:28 dist-test-slave-vkjp krb5kdc[14613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:49:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:49:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:49:28.122920 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:32889
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:44463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:32889
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:28.468554 14628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:28.468858 14628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:28.468950 14628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:28.476133 14628 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:49:28.476248 14628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:28.476296 14628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:28.476357 14628 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:49:28.476406 14628 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:49:28.486336 14628 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:32889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:32889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:28.487722 14628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:28.488998 14628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:28.500962 14636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:28.500998 14637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:28.501029 14639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:28.501959 14628 server_base.cc:1034] running on GCE node
I20260810 14:49:28.502785 14628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:28.504325 14628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:28.505568 14628 hybrid_clock.cc:648] HybridClock initialized: now 1786373368505508 us; error 54 us; skew 500 ppm
I20260810 14:49:28.505913 14628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:28 dist-test-slave-vkjp krb5kdc[14613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373368, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:28.514178 14628 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:49:28.515846 14628 webserver.cc:460] Webserver started at http://127.24.215.190:46521/ using document root <none> and password file <none>
I20260810 14:49:28.516543 14628 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:28.516654 14628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:28.516920 14628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:28.520066 14628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "8f6fca7457f14aeb9e56878a8fbcd10e"
format_stamp: "Formatted at 2026-08-10 14:49:28 on dist-test-slave-vkjp"
server_key: "2eac61108607d2452fb0cdef4c25966a"
server_key_iv: "7668ff8b8f24a76d9856d80960767792"
server_key_version: "encryptionkey@0"
I20260810 14:49:28.520882 14628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "8f6fca7457f14aeb9e56878a8fbcd10e"
format_stamp: "Formatted at 2026-08-10 14:49:28 on dist-test-slave-vkjp"
server_key: "2eac61108607d2452fb0cdef4c25966a"
server_key_iv: "7668ff8b8f24a76d9856d80960767792"
server_key_version: "encryptionkey@0"
I20260810 14:49:28.526257 14628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:49:28.529903 14647 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:28.531533 14628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 14:49:28.531692 14628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "8f6fca7457f14aeb9e56878a8fbcd10e"
format_stamp: "Formatted at 2026-08-10 14:49:28 on dist-test-slave-vkjp"
server_key: "2eac61108607d2452fb0cdef4c25966a"
server_key_iv: "7668ff8b8f24a76d9856d80960767792"
server_key_version: "encryptionkey@0"
I20260810 14:49:28.531908 14628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:28.546938 14628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:28.551971 14628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:28.552294 14628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:28.572144 14628 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:32889
I20260810 14:49:28.572146 14704 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:32889 every 8 connection(s)
I20260810 14:49:28.573613 14628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:49:28.577666 14705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:28.582319 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 14628
I20260810 14:49:28.582552 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:49:28.583166 25438 external_mini_cluster.cc:1428] Setting key 04864b3aac2df86f059ae7c5660fbc40
I20260810 14:49:28.591683 14705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e: Bootstrap starting.
I20260810 14:49:28.595501 14705 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:28.596612 14705 log.cc:824] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:28.599651 14705 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e: No bootstrap required, opened a new log
Aug 10 14:49:28 dist-test-slave-vkjp krb5kdc[14613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:49:28.608023 14705 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6fca7457f14aeb9e56878a8fbcd10e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 32889 } }
I20260810 14:49:28.608486 14705 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:28.608613 14705 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f6fca7457f14aeb9e56878a8fbcd10e, State: Initialized, Role: FOLLOWER
I20260810 14:49:28.608976 14708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:28.585374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34336 (local address 127.24.215.190:32889)
0810 14:49:28.586315 (+ 941us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:28.586334 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:28.586732 (+ 398us) server_negotiation.cc:407] Connection header received
0810 14:49:28.589770 (+ 3038us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:28.589803 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:28.590318 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:28.590781 (+ 463us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:28.592292 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:28.594312 (+ 2020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:28.595822 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:28.596521 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:28.596727 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:28.601119 (+ 4392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:28.601128 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:28.601146 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:28.601190 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:28.604474 (+ 3284us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:28.605231 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:28.605239 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:28.605245 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:28.605417 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:28.605837 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:28.605841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:28.605843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:28.606427 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:28.606783 (+ 356us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:28.607254 (+ 471us) server_negotiation.cc:299] Negotiation successful
0810 14:49:28.607718 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":646,"thread_start_us":372,"threads_started":1}
I20260810 14:49:28.609279 14705 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6fca7457f14aeb9e56878a8fbcd10e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 32889 } }
I20260810 14:49:28.609560 14705 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:49:28.609716 14705 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:49:28.609931 14705 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:28.612593 14705 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6fca7457f14aeb9e56878a8fbcd10e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 32889 } }
I20260810 14:49:28.613085 14705 leader_election.cc:302] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f6fca7457f14aeb9e56878a8fbcd10e; no voters:
I20260810 14:49:28.613646 14705 leader_election.cc:288] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:49:28.613767 14710 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:28.614511 14710 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [term 1 LEADER]: Becoming Leader. State: Replica: 8f6fca7457f14aeb9e56878a8fbcd10e, State: Running, Role: LEADER
I20260810 14:49:28.615121 14710 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6fca7457f14aeb9e56878a8fbcd10e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 32889 } }
I20260810 14:49:28.615872 14705 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:49:28.618932 14712 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f6fca7457f14aeb9e56878a8fbcd10e. Latest consensus state: current_term: 1 leader_uuid: "8f6fca7457f14aeb9e56878a8fbcd10e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6fca7457f14aeb9e56878a8fbcd10e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 32889 } } }
I20260810 14:49:28.619349 14712 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:28.619302 14711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f6fca7457f14aeb9e56878a8fbcd10e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f6fca7457f14aeb9e56878a8fbcd10e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 32889 } } }
I20260810 14:49:28.619614 14711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:28.623700 14716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:49:28.628427 14716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:49:28.637157 14716 catalog_manager.cc:1347] Generated new cluster ID: 36ff127ce02447dfbeab9b13c3b2d99c
I20260810 14:49:28.637266 14716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:49:28.655287 14716 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:49:28.657204 14716 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:49:28.664557 14716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e: Generated new TSK 0
I20260810 14:49:28.665575 14716 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:49:28.733088 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:32889
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:44463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:29.079792 14733 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:29.080143 14733 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:29.080314 14733 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:29.087636 14733 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:29.087797 14733 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:29.087992 14733 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:49:29.098064 14733 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:32889
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:29.099465 14733 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:29.100721 14733 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:29.115098 14739 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:29.114992 14740 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:29.115046 14742 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:29.116011 14733 server_base.cc:1034] running on GCE node
I20260810 14:49:29.116707 14733 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:29.117731 14733 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:29.118965 14733 hybrid_clock.cc:648] HybridClock initialized: now 1786373369118920 us; error 53 us; skew 500 ppm
I20260810 14:49:29.119335 14733 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:29 dist-test-slave-vkjp krb5kdc[14613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373369, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:29.122682 14733 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:49:29.124244 14733 webserver.cc:460] Webserver started at http://127.24.215.129:40697/ using document root <none> and password file <none>
I20260810 14:49:29.124892 14733 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:29.125000 14733 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:29.125270 14733 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:29.128248 14733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "7a60dc25da9045f4a9353cfe8d633721"
format_stamp: "Formatted at 2026-08-10 14:49:29 on dist-test-slave-vkjp"
server_key: "74edfde19b65c012ac1b6090cb6e6966"
server_key_iv: "d391396c54d5c44b3ce0e88a3fa3d4a0"
server_key_version: "encryptionkey@0"
I20260810 14:49:29.129026 14733 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "7a60dc25da9045f4a9353cfe8d633721"
format_stamp: "Formatted at 2026-08-10 14:49:29 on dist-test-slave-vkjp"
server_key: "74edfde19b65c012ac1b6090cb6e6966"
server_key_iv: "d391396c54d5c44b3ce0e88a3fa3d4a0"
server_key_version: "encryptionkey@0"
I20260810 14:49:29.134642 14733 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:49:29.138404 14752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:29.139856 14733 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:49:29.140050 14733 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "7a60dc25da9045f4a9353cfe8d633721"
format_stamp: "Formatted at 2026-08-10 14:49:29 on dist-test-slave-vkjp"
server_key: "74edfde19b65c012ac1b6090cb6e6966"
server_key_iv: "d391396c54d5c44b3ce0e88a3fa3d4a0"
server_key_version: "encryptionkey@0"
I20260810 14:49:29.140264 14733 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:29.159569 14733 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:29.164077 14733 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:29.164448 14733 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:29.165598 14733 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:29.167068 14733 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:29.167232 14733 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260810 14:49:29.167394 14733 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:29.167460 14733 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:29.202533 14733 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:41875
I20260810 14:49:29.202584 14878 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:41875 every 8 connection(s)
I20260810 14:49:29.203886 14733 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:49:29.205417 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 14733
I20260810 14:49:29.205636 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:49:29.206197 25438 external_mini_cluster.cc:1428] Setting key 5ec7d7cbb14fea3886314abae144434c
Aug 10 14:49:29 dist-test-slave-vkjp krb5kdc[14613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373369, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:49:29.228823 14883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:29.206710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:46261 (local address 127.24.215.190:32889)
0810 14:49:29.207216 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:29.207222 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:29.208484 (+ 1262us) server_negotiation.cc:407] Connection header received
0810 14:49:29.211322 (+ 2838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:29.211326 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:29.211391 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:29.211481 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:29.213632 (+ 2151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:29.214336 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:29.216870 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:29.217350 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:29.217445 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:29.221639 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:29.221645 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:29.221648 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:29.221682 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:29.225342 (+ 3660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:29.226101 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:29.226109 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:29.226115 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:29.226264 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:29.226705 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:29.226713 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:29.226719 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:29.227157 (+ 438us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:29.227302 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:29.228035 (+ 733us) server_negotiation.cc:299] Negotiation successful
0810 14:49:29.228332 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":257,"threads_started":1}
I20260810 14:49:29.229420 14884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:29.207364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:32889 (local address 127.24.215.129:46261)
0810 14:49:29.208319 (+ 955us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:29.208360 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:29.211011 (+ 2651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:29.211756 (+ 745us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:29.211769 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:29.212575 (+ 806us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:29.213374 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:29.213402 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:29.214509 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:29.214516 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:29.216735 (+ 2219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:29.216753 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:29.217579 (+ 826us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:29.217589 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:29.217934 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:29.218692 (+ 758us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:49:29.218731 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:29.221410 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:29.225504 (+ 4094us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:29.225512 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:29.225525 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:29.225953 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:29.226382 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:29.226386 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:29.226388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:29.226575 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:29.227290 (+ 715us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:29.227296 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:29.227620 (+ 324us) client_negotiation.cc:769] Sending connection context
0810 14:49:29.227848 (+ 228us) client_negotiation.cc:240] Negotiation successful
0810 14:49:29.228263 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":311,"threads_started":1}
I20260810 14:49:29.231065 14881 heartbeater.cc:347] Connected to a master server at 127.24.215.190:32889
I20260810 14:49:29.231439 14881 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:29.232228 14881 heartbeater.cc:511] Master 127.24.215.190:32889 requested a full tablet report, sending...
I20260810 14:49:29.234428 14668 ts_manager.cc:194] Registered new tserver with Master: 7a60dc25da9045f4a9353cfe8d633721 (127.24.215.129:41875)
I20260810 14:49:29.236111 14668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:46261
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:49:29.274649 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:32889
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:44463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:29.616225 14887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:29.616559 14887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:29.616748 14887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:29.623989 14887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:29.624141 14887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:29.624331 14887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:49:29.634354 14887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:32889
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:29.635828 14887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:29.637238 14887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:29.650699 14892 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:29.650650 14897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:29.650743 14893 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:29.651769 14887 server_base.cc:1034] running on GCE node
I20260810 14:49:29.652562 14887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:29.653513 14887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:29.654760 14887 hybrid_clock.cc:648] HybridClock initialized: now 1786373369654693 us; error 51 us; skew 500 ppm
I20260810 14:49:29.655174 14887 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:29 dist-test-slave-vkjp krb5kdc[14613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373369, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:29.658408 14887 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:49:29.659948 14887 webserver.cc:460] Webserver started at http://127.24.215.130:36781/ using document root <none> and password file <none>
I20260810 14:49:29.660663 14887 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:29.660813 14887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:29.661160 14887 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:29.664141 14887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "1443e3876aa4486883a9c4a81a624fa6"
format_stamp: "Formatted at 2026-08-10 14:49:29 on dist-test-slave-vkjp"
server_key: "f8264c5211416bdb1d1f70c717d5ba27"
server_key_iv: "876f74688cddd7f5688c75bc85962053"
server_key_version: "encryptionkey@0"
I20260810 14:49:29.664929 14887 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "1443e3876aa4486883a9c4a81a624fa6"
format_stamp: "Formatted at 2026-08-10 14:49:29 on dist-test-slave-vkjp"
server_key: "f8264c5211416bdb1d1f70c717d5ba27"
server_key_iv: "876f74688cddd7f5688c75bc85962053"
server_key_version: "encryptionkey@0"
I20260810 14:49:29.670207 14887 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:49:29.673676 14905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:29.674969 14887 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:49:29.675195 14887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "1443e3876aa4486883a9c4a81a624fa6"
format_stamp: "Formatted at 2026-08-10 14:49:29 on dist-test-slave-vkjp"
server_key: "f8264c5211416bdb1d1f70c717d5ba27"
server_key_iv: "876f74688cddd7f5688c75bc85962053"
server_key_version: "encryptionkey@0"
I20260810 14:49:29.675408 14887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:29.687740 14887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:29.691447 14887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:29.691737 14887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:29.692695 14887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:29.694022 14887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:29.694126 14887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:29.694273 14887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:29.694344 14887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:29.728046 14887 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:33631
I20260810 14:49:29.728097 15038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:33631 every 8 connection(s)
I20260810 14:49:29.729386 14887 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:49:29.734162 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 14887
I20260810 14:49:29.734367 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:49:29.734808 25438 external_mini_cluster.cc:1428] Setting key d20c66783b6b41f137355aed3dff900d
Aug 10 14:49:29 dist-test-slave-vkjp krb5kdc[14613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373369, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:49:29.751796 15041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:29.732113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:33277 (local address 127.24.215.190:32889)
0810 14:49:29.732578 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:29.732583 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:29.733590 (+ 1007us) server_negotiation.cc:407] Connection header received
0810 14:49:29.735600 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:29.735604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:29.735698 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:29.735823 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:29.737694 (+ 1871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:29.738868 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:29.740424 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:29.741211 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:29.741354 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:29.745144 (+ 3790us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:29.745153 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:29.745161 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:29.745207 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:29.748247 (+ 3040us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:29.748987 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:29.748992 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:29.748994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:29.749194 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:29.749680 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:29.749687 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:29.749692 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:29.750103 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:29.750271 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:29.750988 (+ 717us) server_negotiation.cc:299] Negotiation successful
0810 14:49:29.751293 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":233,"threads_started":1}
I20260810 14:49:29.752522 15042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:29.732668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:32889 (local address 127.24.215.130:33277)
0810 14:49:29.733425 (+ 757us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:29.733469 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:29.735312 (+ 1843us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:29.736068 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:29.736078 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:29.736824 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:29.737518 (+ 694us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:29.737537 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:29.739007 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:29.739011 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:29.740275 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:29.740287 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:29.741414 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:29.741419 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:29.741613 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:29.742299 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:49:29.742320 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:29.744784 (+ 2464us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:29.748397 (+ 3613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:29.748406 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:29.748425 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:29.748847 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:29.749358 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:29.749364 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:29.749367 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:29.749554 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:29.750231 (+ 677us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:29.750237 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:29.750606 (+ 369us) client_negotiation.cc:769] Sending connection context
0810 14:49:29.750828 (+ 222us) client_negotiation.cc:240] Negotiation successful
0810 14:49:29.751364 (+ 536us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"thread_start_us":325,"threads_started":1}
I20260810 14:49:29.754032 15039 heartbeater.cc:347] Connected to a master server at 127.24.215.190:32889
I20260810 14:49:29.754356 15039 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:29.755146 15039 heartbeater.cc:511] Master 127.24.215.190:32889 requested a full tablet report, sending...
I20260810 14:49:29.756604 14668 ts_manager.cc:194] Registered new tserver with Master: 1443e3876aa4486883a9c4a81a624fa6 (127.24.215.130:33631)
I20260810 14:49:29.757405 14668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:33277
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:49:29.802681 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:32889
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:44463
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:30.135792 15045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:30.136087 15045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:30.136219 15045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:30.143527 15045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:30.143667 15045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:30.143847 15045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:49:30.153796 15045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44463
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:32889
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:30.155160 15045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:30.156461 15045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:30.169888 15053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:30.170006 15051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:30.170230 15050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:30.171056 15045 server_base.cc:1034] running on GCE node
I20260810 14:49:30.171715 15045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:30.172657 15045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:30.173884 15045 hybrid_clock.cc:648] HybridClock initialized: now 1786373370173851 us; error 38 us; skew 500 ppm
I20260810 14:49:30.174199 15045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:30 dist-test-slave-vkjp krb5kdc[14613](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373370, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:30.177459 15045 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:49:30.179014 15045 webserver.cc:460] Webserver started at http://127.24.215.131:36805/ using document root <none> and password file <none>
I20260810 14:49:30.179747 15045 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:30.179911 15045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:30.180236 15045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:30.183061 15045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "08b2b6cd2ca04d319c144428732dc3c5"
format_stamp: "Formatted at 2026-08-10 14:49:30 on dist-test-slave-vkjp"
server_key: "f38f8c66308240a616eb346bdb4aa439"
server_key_iv: "7d54155df50845b97cf17b93d94a21ea"
server_key_version: "encryptionkey@0"
I20260810 14:49:30.183885 15045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "08b2b6cd2ca04d319c144428732dc3c5"
format_stamp: "Formatted at 2026-08-10 14:49:30 on dist-test-slave-vkjp"
server_key: "f38f8c66308240a616eb346bdb4aa439"
server_key_iv: "7d54155df50845b97cf17b93d94a21ea"
server_key_version: "encryptionkey@0"
I20260810 14:49:30.188920 15045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:49:30.192734 15061 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:30.194571 15045 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260810 14:49:30.194752 15045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "08b2b6cd2ca04d319c144428732dc3c5"
format_stamp: "Formatted at 2026-08-10 14:49:30 on dist-test-slave-vkjp"
server_key: "f38f8c66308240a616eb346bdb4aa439"
server_key_iv: "7d54155df50845b97cf17b93d94a21ea"
server_key_version: "encryptionkey@0"
I20260810 14:49:30.195012 15045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:30.225032 15045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:30.229749 15045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:30.230031 15045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:30.231051 15045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:30.232714 15045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:30.232801 15045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:30.232908 15045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:30.232959 15045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:30.239094 14881 heartbeater.cc:503] Master 127.24.215.190:32889 was elected leader, sending a full tablet report...
I20260810 14:49:30.271652 15045 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:38915
I20260810 14:49:30.271711 15188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:38915 every 8 connection(s)
I20260810 14:49:30.272984 15045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:49:30.282680 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 15045
I20260810 14:49:30.282917 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:49:30.283520 25438 external_mini_cluster.cc:1428] Setting key d9a5a64c1aa86a8c3cc11e41f1608e13
Aug 10 14:49:30 dist-test-slave-vkjp krb5kdc[14613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373370, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:49:30.295300 15191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.275811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:39459 (local address 127.24.215.190:32889)
0810 14:49:30.276354 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:30.276359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:30.277332 (+ 973us) server_negotiation.cc:407] Connection header received
0810 14:49:30.279426 (+ 2094us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:30.279430 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:30.279529 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:30.279628 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:30.281409 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.282207 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.284591 (+ 2384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.285175 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.285278 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:30.289386 (+ 4108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:30.289391 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:30.289394 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:30.289428 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:30.292264 (+ 2836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:30.293033 (+ 769us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:30.293037 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:30.293039 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:30.293157 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:30.293639 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:30.293642 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:30.293644 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:30.293906 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:30.294033 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:30.294769 (+ 736us) server_negotiation.cc:299] Negotiation successful
0810 14:49:30.294937 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":249,"threads_started":1}
I20260810 14:49:30.296464 15192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.276412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:32889 (local address 127.24.215.131:39459)
0810 14:49:30.277158 (+ 746us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:30.277200 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:30.279156 (+ 1956us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:30.279908 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:30.279917 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:30.280549 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:30.281146 (+ 597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:30.281166 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.282427 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.282434 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:30.284431 (+ 1997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:30.284447 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.285484 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.285490 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:30.285854 (+ 364us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:30.286672 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:49:30.286706 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:30.289178 (+ 2472us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:30.292410 (+ 3232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:30.292418 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:30.292431 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:30.292913 (+ 482us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:30.293283 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:30.293320 (+ 37us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:30.293322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:30.293532 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:30.294039 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:30.294045 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:30.294421 (+ 376us) client_negotiation.cc:769] Sending connection context
0810 14:49:30.294665 (+ 244us) client_negotiation.cc:240] Negotiation successful
0810 14:49:30.295069 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":304,"threads_started":1}
I20260810 14:49:30.298166 15189 heartbeater.cc:347] Connected to a master server at 127.24.215.190:32889
I20260810 14:49:30.298549 15189 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:30.299525 15189 heartbeater.cc:511] Master 127.24.215.190:32889 requested a full tablet report, sending...
I20260810 14:49:30.300954 14668 ts_manager.cc:194] Registered new tserver with Master: 08b2b6cd2ca04d319c144428732dc3c5 (127.24.215.131:38915)
I20260810 14:49:30.301800 14668 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:39459
I20260810 14:49:30.311971 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:49:30.332657 15191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.316109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34344 (local address 127.24.215.190:32889)
0810 14:49:30.316365 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:30.316371 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:30.316700 (+ 329us) server_negotiation.cc:407] Connection header received
0810 14:49:30.316888 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:30.316892 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:30.316955 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:30.317042 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:30.318493 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.319298 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.320611 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.320996 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.321096 (+ 100us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:30.327044 (+ 5948us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:30.327049 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:30.327052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:30.327160 (+ 108us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:30.329816 (+ 2656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:30.330542 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:30.330545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:30.330547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:30.330659 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:30.331047 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:30.331050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:30.331052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:30.331508 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:30.331723 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:30.332137 (+ 414us) server_negotiation.cc:299] Negotiation successful
0810 14:49:30.332298 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:49:30.336619 14668 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:49:30.343839 14668 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:49:30.389084 15203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.356666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:41875 (local address 127.0.0.1:37894)
0810 14:49:30.379823 (+ 23157us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:30.379844 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:30.379962 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:30.381070 (+ 1108us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:30.381079 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:30.381112 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:30.381706 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:30.381724 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.383575 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.383584 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:30.385869 (+ 2285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:30.385883 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.387949 (+ 2066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.387955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:30.388175 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:30.388232 (+ 57us) client_negotiation.cc:769] Sending connection context
0810 14:49:30.388368 (+ 136us) client_negotiation.cc:240] Negotiation successful
0810 14:49:30.388589 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22155,"mutex_wait_us":29,"thread_start_us":24949,"threads_started":1}
I20260810 14:49:30.389495 15205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.357352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53106 (local address 127.24.215.130:33631)
0810 14:49:30.381056 (+ 23704us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:30.381064 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:30.381089 (+ 25us) server_negotiation.cc:407] Connection header received
0810 14:49:30.381182 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:30.381190 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:30.381405 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:30.381598 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:30.382564 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.384107 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.386612 (+ 2505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.388119 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.388288 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:30.388451 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:30.388835 (+ 384us) server_negotiation.cc:299] Negotiation successful
0810 14:49:30.389022 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23488,"thread_start_us":23757,"threads_started":1}
I20260810 14:49:30.389719 15206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.356739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:33631 (local address 127.0.0.1:53106)
0810 14:49:30.379547 (+ 22808us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:30.379595 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:30.379797 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:30.381834 (+ 2037us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:30.381838 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:30.381867 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:30.382328 (+ 461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:30.382341 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.384264 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.384269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:30.386439 (+ 2170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:30.386453 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.388443 (+ 1990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.388449 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:30.388640 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:30.388670 (+ 30us) client_negotiation.cc:769] Sending connection context
0810 14:49:30.388773 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 14:49:30.388892 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22681,"mutex_wait_us":53,"thread_start_us":21922,"threads_started":1}
I20260810 14:49:30.389709 15204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.356806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37894 (local address 127.24.215.129:41875)
0810 14:49:30.375781 (+ 18975us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:30.375794 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:30.380084 (+ 4290us) server_negotiation.cc:407] Connection header received
0810 14:49:30.380196 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:30.380217 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:30.380513 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:30.380726 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:30.381905 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.383447 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.386055 (+ 2608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.387400 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.387615 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:30.387803 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:30.389195 (+ 1392us) server_negotiation.cc:299] Negotiation successful
0810 14:49:30.389382 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18706,"thread_start_us":18280,"threads_started":1}
I20260810 14:49:30.390244 15202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.359968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:38915 (local address 127.0.0.1:42876)
0810 14:49:30.381904 (+ 21936us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:30.381922 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:30.382063 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:30.382641 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:30.382646 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:30.382671 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:30.383002 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:30.383013 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.384760 (+ 1747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.384765 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:30.386549 (+ 1784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:30.386564 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.388601 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.388606 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:30.388760 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:30.388795 (+ 35us) client_negotiation.cc:769] Sending connection context
0810 14:49:30.388884 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 14:49:30.389030 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21813,"mutex_wait_us":26,"thread_start_us":20458,"threads_started":1}
I20260810 14:49:30.390681 15207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.360326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42876 (local address 127.24.215.131:38915)
0810 14:49:30.379620 (+ 19294us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:30.379631 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:30.382042 (+ 2411us) server_negotiation.cc:407] Connection header received
0810 14:49:30.382219 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:30.382226 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:30.382453 (+ 227us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:30.382634 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:30.383208 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.384614 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.386734 (+ 2120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.388162 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.388326 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:30.388451 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:30.389175 (+ 724us) server_negotiation.cc:299] Negotiation successful
0810 14:49:30.389514 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19110,"thread_start_us":19291,"threads_started":1}
I20260810 14:49:30.393229 14804 tablet_service.cc:1467] Processing CreateTablet for tablet b0cb857772d9483888e80b0e9fdbd210 (DEFAULT_TABLE table=test-table [id=410af2e3c3494fad94b69f7f608d0def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:30.393489 14964 tablet_service.cc:1467] Processing CreateTablet for tablet b0cb857772d9483888e80b0e9fdbd210 (DEFAULT_TABLE table=test-table [id=410af2e3c3494fad94b69f7f608d0def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:30.394049 15113 tablet_service.cc:1467] Processing CreateTablet for tablet b0cb857772d9483888e80b0e9fdbd210 (DEFAULT_TABLE table=test-table [id=410af2e3c3494fad94b69f7f608d0def]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:30.394603 14804 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0cb857772d9483888e80b0e9fdbd210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:30.394860 14964 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0cb857772d9483888e80b0e9fdbd210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:30.395434 15113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0cb857772d9483888e80b0e9fdbd210. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:30.406332 15208 tablet_bootstrap.cc:492] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721: Bootstrap starting.
I20260810 14:49:30.410157 15210 tablet_bootstrap.cc:492] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6: Bootstrap starting.
I20260810 14:49:30.411341 15209 tablet_bootstrap.cc:492] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5: Bootstrap starting.
I20260810 14:49:30.411761 15208 tablet_bootstrap.cc:654] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:30.413266 15208 log.cc:824] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:30.414512 15210 tablet_bootstrap.cc:654] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:30.414619 15209 tablet_bootstrap.cc:654] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:30.416186 15210 log.cc:824] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:30.416335 15209 log.cc:824] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:30.419772 15209 tablet_bootstrap.cc:492] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5: No bootstrap required, opened a new log
I20260810 14:49:30.419878 15208 tablet_bootstrap.cc:492] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721: No bootstrap required, opened a new log
I20260810 14:49:30.419979 15210 tablet_bootstrap.cc:492] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6: No bootstrap required, opened a new log
I20260810 14:49:30.420318 15210 ts_tablet_manager.cc:1397] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6: Time spent bootstrapping tablet: real 0.010s user 0.003s sys 0.006s
I20260810 14:49:30.420281 15209 ts_tablet_manager.cc:1397] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.003s
I20260810 14:49:30.420707 15208 ts_tablet_manager.cc:1397] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721: Time spent bootstrapping tablet: real 0.015s user 0.006s sys 0.005s
I20260810 14:49:30.427980 15209 raft_consensus.cc:348] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a60dc25da9045f4a9353cfe8d633721" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41875 } } peers { permanent_uuid: "1443e3876aa4486883a9c4a81a624fa6" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33631 } } peers { permanent_uuid: "08b2b6cd2ca04d319c144428732dc3c5" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38915 } }
I20260810 14:49:30.428386 15209 raft_consensus.cc:374] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:30.428495 15209 raft_consensus.cc:729] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 08b2b6cd2ca04d319c144428732dc3c5, State: Initialized, Role: FOLLOWER
I20260810 14:49:30.429134 15209 consensus_queue.cc:260] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a60dc25da9045f4a9353cfe8d633721" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41875 } } peers { permanent_uuid: "1443e3876aa4486883a9c4a81a624fa6" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33631 } } peers { permanent_uuid: "08b2b6cd2ca04d319c144428732dc3c5" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38915 } }
I20260810 14:49:30.430284 15189 heartbeater.cc:503] Master 127.24.215.190:32889 was elected leader, sending a full tablet report...
I20260810 14:49:30.430998 15209 ts_tablet_manager.cc:1428] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5: Time spent starting tablet: real 0.011s user 0.009s sys 0.000s
I20260810 14:49:30.431715 15210 raft_consensus.cc:348] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a60dc25da9045f4a9353cfe8d633721" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41875 } } peers { permanent_uuid: "1443e3876aa4486883a9c4a81a624fa6" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33631 } } peers { permanent_uuid: "08b2b6cd2ca04d319c144428732dc3c5" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38915 } }
I20260810 14:49:30.431718 15208 raft_consensus.cc:348] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a60dc25da9045f4a9353cfe8d633721" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41875 } } peers { permanent_uuid: "1443e3876aa4486883a9c4a81a624fa6" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33631 } } peers { permanent_uuid: "08b2b6cd2ca04d319c144428732dc3c5" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38915 } }
I20260810 14:49:30.432178 15208 raft_consensus.cc:374] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:30.432178 15210 raft_consensus.cc:374] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:30.432286 15208 raft_consensus.cc:729] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a60dc25da9045f4a9353cfe8d633721, State: Initialized, Role: FOLLOWER
I20260810 14:49:30.432286 15210 raft_consensus.cc:729] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1443e3876aa4486883a9c4a81a624fa6, State: Initialized, Role: FOLLOWER
I20260810 14:49:30.432899 15208 consensus_queue.cc:260] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a60dc25da9045f4a9353cfe8d633721" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41875 } } peers { permanent_uuid: "1443e3876aa4486883a9c4a81a624fa6" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33631 } } peers { permanent_uuid: "08b2b6cd2ca04d319c144428732dc3c5" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38915 } }
I20260810 14:49:30.432900 15210 consensus_queue.cc:260] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a60dc25da9045f4a9353cfe8d633721" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41875 } } peers { permanent_uuid: "1443e3876aa4486883a9c4a81a624fa6" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33631 } } peers { permanent_uuid: "08b2b6cd2ca04d319c144428732dc3c5" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38915 } }
I20260810 14:49:30.434769 15210 ts_tablet_manager.cc:1428] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6: Time spent starting tablet: real 0.014s user 0.011s sys 0.001s
I20260810 14:49:30.435176 15208 ts_tablet_manager.cc:1428] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721: Time spent starting tablet: real 0.014s user 0.011s sys 0.004s
I20260810 14:49:30.435609 15039 heartbeater.cc:503] Master 127.24.215.190:32889 was elected leader, sending a full tablet report...
W20260810 14:49:30.456367 14882 tablet.cc:2369] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:49:30.481418 15040 tablet.cc:2369] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:49:30.524695 15190 tablet.cc:2369] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:49:30.614573 15217 raft_consensus.cc:482] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:49:30.615031 15217 raft_consensus.cc:504] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a60dc25da9045f4a9353cfe8d633721" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41875 } } peers { permanent_uuid: "1443e3876aa4486883a9c4a81a624fa6" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33631 } } peers { permanent_uuid: "08b2b6cd2ca04d319c144428732dc3c5" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38915 } }
I20260810 14:49:30.617005 15217 leader_election.cc:288] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1443e3876aa4486883a9c4a81a624fa6 (127.24.215.130:33631), 08b2b6cd2ca04d319c144428732dc3c5 (127.24.215.131:38915)
I20260810 14:49:30.624357 15207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.617554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:47037 (local address 127.24.215.131:38915)
0810 14:49:30.617866 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:30.617876 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:30.618846 (+ 970us) server_negotiation.cc:407] Connection header received
0810 14:49:30.619070 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:30.619114 (+ 44us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:30.619203 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:30.619304 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:30.619898 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.620647 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.622524 (+ 1877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.623384 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.623581 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:30.623659 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:30.623952 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 14:49:30.624085 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96}
I20260810 14:49:30.624500 15218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.617284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:38915 (local address 127.24.215.129:47037)
0810 14:49:30.618724 (+ 1440us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:30.618741 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:30.618925 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:30.619326 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:30.619331 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:30.619356 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:30.619748 (+ 392us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:30.619758 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.620787 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.620793 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:30.622351 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:30.622364 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.623601 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.623607 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:30.623783 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:30.623808 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 14:49:30.623927 (+ 119us) client_negotiation.cc:240] Negotiation successful
0810 14:49:30.624088 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1319,"mutex_wait_us":1,"thread_start_us":811,"threads_started":1}
I20260810 14:49:30.624815 15205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.617590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:33105 (local address 127.24.215.130:33631)
0810 14:49:30.617862 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:30.617870 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:30.618929 (+ 1059us) server_negotiation.cc:407] Connection header received
0810 14:49:30.619180 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:30.619185 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:30.619252 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:30.619377 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:30.620092 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.620841 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.622740 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.623698 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.623898 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:30.623993 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:30.624329 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 14:49:30.624472 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 14:49:30.625069 15219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.617284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:33631 (local address 127.24.215.129:33105)
0810 14:49:30.618827 (+ 1543us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:30.618843 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:30.618976 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:30.619391 (+ 415us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:30.619396 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:30.619427 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:30.619922 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:30.619932 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.621009 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.621036 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:30.622561 (+ 1525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:30.622576 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.623960 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.623967 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:30.624143 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:30.624168 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 14:49:30.624295 (+ 127us) client_negotiation.cc:240] Negotiation successful
0810 14:49:30.624541 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1406,"mutex_wait_us":89,"thread_start_us":983,"threads_started":1}
I20260810 14:49:30.625591 15135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0cb857772d9483888e80b0e9fdbd210" candidate_uuid: "7a60dc25da9045f4a9353cfe8d633721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08b2b6cd2ca04d319c144428732dc3c5" is_pre_election: true
I20260810 14:49:30.626245 15135 raft_consensus.cc:2454] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a60dc25da9045f4a9353cfe8d633721 in term 0.
I20260810 14:49:30.626255 14988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0cb857772d9483888e80b0e9fdbd210" candidate_uuid: "7a60dc25da9045f4a9353cfe8d633721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1443e3876aa4486883a9c4a81a624fa6" is_pre_election: true
I20260810 14:49:30.626865 14988 raft_consensus.cc:2454] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7a60dc25da9045f4a9353cfe8d633721 in term 0.
I20260810 14:49:30.627112 14753 leader_election.cc:302] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08b2b6cd2ca04d319c144428732dc3c5, 7a60dc25da9045f4a9353cfe8d633721; no voters:
I20260810 14:49:30.627606 15217 raft_consensus.cc:2781] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:49:30.627776 15217 raft_consensus.cc:482] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:49:30.627913 15217 raft_consensus.cc:3037] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:30.630155 15217 raft_consensus.cc:504] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a60dc25da9045f4a9353cfe8d633721" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41875 } } peers { permanent_uuid: "1443e3876aa4486883a9c4a81a624fa6" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33631 } } peers { permanent_uuid: "08b2b6cd2ca04d319c144428732dc3c5" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38915 } }
I20260810 14:49:30.630780 15217 leader_election.cc:288] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [CANDIDATE]: Term 1 election: Requested vote from peers 1443e3876aa4486883a9c4a81a624fa6 (127.24.215.130:33631), 08b2b6cd2ca04d319c144428732dc3c5 (127.24.215.131:38915)
I20260810 14:49:30.631351 14988 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0cb857772d9483888e80b0e9fdbd210" candidate_uuid: "7a60dc25da9045f4a9353cfe8d633721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1443e3876aa4486883a9c4a81a624fa6"
I20260810 14:49:30.631361 15135 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0cb857772d9483888e80b0e9fdbd210" candidate_uuid: "7a60dc25da9045f4a9353cfe8d633721" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "08b2b6cd2ca04d319c144428732dc3c5"
I20260810 14:49:30.631628 15135 raft_consensus.cc:3037] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:30.631635 14988 raft_consensus.cc:3037] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:30.633949 15135 raft_consensus.cc:2454] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a60dc25da9045f4a9353cfe8d633721 in term 1.
I20260810 14:49:30.633980 14988 raft_consensus.cc:2454] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7a60dc25da9045f4a9353cfe8d633721 in term 1.
I20260810 14:49:30.634418 14753 leader_election.cc:302] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 08b2b6cd2ca04d319c144428732dc3c5, 7a60dc25da9045f4a9353cfe8d633721; no voters:
I20260810 14:49:30.634743 15217 raft_consensus.cc:2781] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:30.635227 15217 raft_consensus.cc:686] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [term 1 LEADER]: Becoming Leader. State: Replica: 7a60dc25da9045f4a9353cfe8d633721, State: Running, Role: LEADER
I20260810 14:49:30.635763 15217 consensus_queue.cc:237] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a60dc25da9045f4a9353cfe8d633721" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41875 } } peers { permanent_uuid: "1443e3876aa4486883a9c4a81a624fa6" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33631 } } peers { permanent_uuid: "08b2b6cd2ca04d319c144428732dc3c5" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38915 } }
I20260810 14:49:30.640527 14665 catalog_manager.cc:5495] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7a60dc25da9045f4a9353cfe8d633721 (127.24.215.129). New cstate: current_term: 1 leader_uuid: "7a60dc25da9045f4a9353cfe8d633721" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7a60dc25da9045f4a9353cfe8d633721" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 41875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "1443e3876aa4486883a9c4a81a624fa6" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "08b2b6cd2ca04d319c144428732dc3c5" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38915 } health_report { overall_health: UNKNOWN } } }
I20260810 14:49:30.671819 15204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.666210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37898 (local address 127.24.215.129:41875)
0810 14:49:30.666430 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:30.666436 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:30.666456 (+ 20us) server_negotiation.cc:407] Connection header received
0810 14:49:30.666661 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:30.666665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:30.666742 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:30.666899 (+ 157us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:49:30.667386 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.668100 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.669500 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.669912 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.670051 (+ 139us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:30.670695 (+ 644us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:49:30.671209 (+ 514us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:49:30.671408 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:30.671540 (+ 132us) server_negotiation.cc:299] Negotiation successful
0810 14:49:30.671590 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 14:49:30.680217 14988 raft_consensus.cc:1261] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6 [term 1 FOLLOWER]: Refusing update from remote peer 7a60dc25da9045f4a9353cfe8d633721: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:49:30.680217 15135 raft_consensus.cc:1261] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5 [term 1 FOLLOWER]: Refusing update from remote peer 7a60dc25da9045f4a9353cfe8d633721: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:49:30.681360 15220 consensus_queue.cc:1035] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "08b2b6cd2ca04d319c144428732dc3c5" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 38915 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:49:30.681735 15217 consensus_queue.cc:1035] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1443e3876aa4486883a9c4a81a624fa6" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 33631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:49:30.706707 14665 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:49:30.706991 14665 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:49:30.710788 14665 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f6fca7457f14aeb9e56878a8fbcd10e: Sending DeleteTablet for 3 replicas of tablet b0cb857772d9483888e80b0e9fdbd210
I20260810 14:49:30.712018 14804 tablet_service.cc:1514] Processing DeleteTablet for tablet b0cb857772d9483888e80b0e9fdbd210 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:30 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:37894
I20260810 14:49:30.712167 15113 tablet_service.cc:1514] Processing DeleteTablet for tablet b0cb857772d9483888e80b0e9fdbd210 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:30 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:42876
I20260810 14:49:30.713176 15234 tablet_replica.cc:330] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721: stopping tablet replica
I20260810 14:49:30.713506 14964 tablet_service.cc:1514] Processing DeleteTablet for tablet b0cb857772d9483888e80b0e9fdbd210 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:30 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:53106
I20260810 14:49:30.713819 15234 raft_consensus.cc:2229] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:49:30.714573 15234 raft_consensus.cc:2258] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:30.716089 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 14733
I20260810 14:49:30.716672 15234 ts_tablet_manager.cc:1905] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:30.724371 15234 ts_tablet_manager.cc:1918] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:49:30.724674 15234 log.cc:1196] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/b0cb857772d9483888e80b0e9fdbd210
I20260810 14:49:30.725315 15234 ts_tablet_manager.cc:1939] T b0cb857772d9483888e80b0e9fdbd210 P 7a60dc25da9045f4a9353cfe8d633721: Deleting consensus metadata
I20260810 14:49:30.726126 15235 tablet_replica.cc:330] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5: stopping tablet replica
I20260810 14:49:30.726656 15235 raft_consensus.cc:2229] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:49:30.726779 15236 tablet_replica.cc:330] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6: stopping tablet replica
I20260810 14:49:30.727231 15235 raft_consensus.cc:2258] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:30.727200 15236 raft_consensus.cc:2229] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:49:30.727898 15236 raft_consensus.cc:2258] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:30.728685 14653 catalog_manager.cc:4841] TS 7a60dc25da9045f4a9353cfe8d633721 (127.24.215.129:41875): tablet b0cb857772d9483888e80b0e9fdbd210 (table test-table [id=410af2e3c3494fad94b69f7f608d0def]) successfully deleted
I20260810 14:49:30.729538 15235 ts_tablet_manager.cc:1905] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:30.730355 15236 ts_tablet_manager.cc:1905] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:30.736773 15235 ts_tablet_manager.cc:1918] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:49:30.736948 15235 log.cc:1196] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/b0cb857772d9483888e80b0e9fdbd210
I20260810 14:49:30.737462 15235 ts_tablet_manager.cc:1939] T b0cb857772d9483888e80b0e9fdbd210 P 08b2b6cd2ca04d319c144428732dc3c5: Deleting consensus metadata
Aug 10 14:49:30 dist-test-slave-vkjp krb5kdc[14613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:49:30.739380 14650 catalog_manager.cc:4841] TS 08b2b6cd2ca04d319c144428732dc3c5 (127.24.215.131:38915): tablet b0cb857772d9483888e80b0e9fdbd210 (table test-table [id=410af2e3c3494fad94b69f7f608d0def]) successfully deleted
I20260810 14:49:30.742142 15236 ts_tablet_manager.cc:1918] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:49:30.742626 15236 log.cc:1196] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/b0cb857772d9483888e80b0e9fdbd210
I20260810 14:49:30.743243 15236 ts_tablet_manager.cc:1939] T b0cb857772d9483888e80b0e9fdbd210 P 1443e3876aa4486883a9c4a81a624fa6: Deleting consensus metadata
I20260810 14:49:30.745234 14654 catalog_manager.cc:4841] TS 1443e3876aa4486883a9c4a81a624fa6 (127.24.215.130:33631): tablet b0cb857772d9483888e80b0e9fdbd210 (table test-table [id=410af2e3c3494fad94b69f7f608d0def]) successfully deleted
I20260810 14:49:30.747681 15204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.719290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37912 (local address 127.24.215.129:41875)
0810 14:49:30.719903 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:30.719909 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:30.721546 (+ 1637us) server_negotiation.cc:407] Connection header received
0810 14:49:30.721801 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:30.721809 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:30.721902 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:30.722028 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:30.730144 (+ 8116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.731329 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.733534 (+ 2205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.734309 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.734417 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:30.740019 (+ 5602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:30.740035 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:30.740054 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:30.740107 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:30.744456 (+ 4349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:30.745438 (+ 982us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:30.745447 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:30.745452 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:30.745643 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:30.746065 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:30.746070 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:30.746074 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:30.746421 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:30.746768 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:30.747061 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 14:49:30.747351 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499}
I20260810 14:49:30.748497 14874 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:30.849040 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 14733
I20260810 14:49:30.871883 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 14887
Aug 10 14:49:30 dist-test-slave-vkjp krb5kdc[14613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:49:30.886869 15205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.872666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53110 (local address 127.24.215.130:33631)
0810 14:49:30.872856 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:30.872861 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:30.872879 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:49:30.873023 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:30.873027 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:30.873089 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:30.873173 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:30.874423 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.875208 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.876535 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:30.876892 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:30.876988 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:30.881433 (+ 4445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:30.881445 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:30.881462 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:30.881497 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:30.884302 (+ 2805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:30.885024 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:30.885031 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:30.885033 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:30.885156 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:30.885473 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:30.885476 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:30.885477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:30.885853 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:30.886083 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:30.886332 (+ 249us) server_negotiation.cc:299] Negotiation successful
0810 14:49:30.886493 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 14:49:30.887537 15033 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:30.977419 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 14887
I20260810 14:49:30.998617 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 15045
Aug 10 14:49:31 dist-test-slave-vkjp krb5kdc[14613](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373368, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:49:31.014348 15207 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:30.999503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42884 (local address 127.24.215.131:38915)
0810 14:49:30.999713 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:30.999720 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:30.999744 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:49:30.999820 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:30.999831 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:30.999907 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:31.000019 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:31.001420 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:31.002159 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:31.003624 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:31.004027 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:31.004162 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:31.008912 (+ 4750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:31.008930 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:31.008948 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:31.008986 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:31.011770 (+ 2784us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:31.012503 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:31.012511 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:31.012513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:31.012644 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:31.013002 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:31.013006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:31.013008 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:31.013358 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:31.013586 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:31.013918 (+ 332us) server_negotiation.cc:299] Negotiation successful
0810 14:49:31.014085 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 14:49:31.015431 15182 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:31.105252 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 15045
I20260810 14:49:31.126358 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 14628
I20260810 14:49:31.127553 14700 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:31.238497 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 14628
2026-08-10T14:49:31Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4634 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-10T14:49:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:49:31Z Disabled control of system clock
I20260810 14:49:31.329972 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:39481
--webserver_interface=127.24.215.190
--webserver_port=0
--builtin_ntp_servers=127.24.215.148:41789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:39481
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:49:31.675266 15247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:31.675545 15247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:31.675637 15247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:31.682852 15247 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:49:31.682956 15247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:31.682996 15247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:31.683053 15247 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:49:31.683161 15247 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:49:31.692365 15247 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:41789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:39481
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:39481
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:31.693583 15247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:31.695494 15247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:31.707680 15256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:31.707836 15253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:31.707710 15254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:31.709216 15247 server_base.cc:1034] running on GCE node
I20260810 14:49:31.710033 15247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:31.711520 15247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:31.712744 15247 hybrid_clock.cc:648] HybridClock initialized: now 1786373371712708 us; error 55 us; skew 500 ppm
I20260810 14:49:31.713101 15247 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:49:31.715497 15247 webserver.cc:460] Webserver started at http://127.24.215.190:38687/ using document root <none> and password file <none>
I20260810 14:49:31.716181 15247 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:31.716313 15247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:31.716656 15247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:31.719632 15247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "9b74c52694d64d7884569e86bab773b9"
format_stamp: "Formatted at 2026-08-10 14:49:31 on dist-test-slave-vkjp"
server_key: "92f9a0ea5484c04a280e1b05e217a95e"
server_key_iv: "5da66870074c3e55e10658852de172d8"
server_key_version: "encryptionkey@0"
I20260810 14:49:31.720449 15247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "9b74c52694d64d7884569e86bab773b9"
format_stamp: "Formatted at 2026-08-10 14:49:31 on dist-test-slave-vkjp"
server_key: "92f9a0ea5484c04a280e1b05e217a95e"
server_key_iv: "5da66870074c3e55e10658852de172d8"
server_key_version: "encryptionkey@0"
I20260810 14:49:31.725695 15247 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:49:31.729321 15262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:31.730681 15247 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:49:31.730865 15247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "9b74c52694d64d7884569e86bab773b9"
format_stamp: "Formatted at 2026-08-10 14:49:31 on dist-test-slave-vkjp"
server_key: "92f9a0ea5484c04a280e1b05e217a95e"
server_key_iv: "5da66870074c3e55e10658852de172d8"
server_key_version: "encryptionkey@0"
I20260810 14:49:31.731086 15247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:31.746917 15247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:31.747761 15247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:31.748056 15247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:31.767702 15247 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:39481
I20260810 14:49:31.767740 15321 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:39481 every 8 connection(s)
I20260810 14:49:31.769073 15247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:49:31.772886 15322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:31.776254 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 15247
I20260810 14:49:31.776434 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:49:31.776991 25438 external_mini_cluster.cc:1428] Setting key b8d38ac07eaeea600224312fc83d8374
I20260810 14:49:31.785135 15322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9: Bootstrap starting.
I20260810 14:49:31.788390 15322 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:31.789525 15322 log.cc:824] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:31.790302 15325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:31.778961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56302 (local address 127.24.215.190:39481)
0810 14:49:31.779976 (+ 1015us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:31.779994 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:31.780034 (+ 40us) server_negotiation.cc:407] Connection header received
0810 14:49:31.781689 (+ 1655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:31.781711 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:31.782172 (+ 461us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:31.782567 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:31.783208 (+ 641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:31.784442 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:31.785874 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:31.786537 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:31.786752 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:31.787926 (+ 1174us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:31.787934 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:31.787947 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:31.788476 (+ 529us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:31.788619 (+ 143us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:31.788642 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:31.788775 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:31.789005 (+ 230us) server_negotiation.cc:299] Negotiation successful
0810 14:49:31.789293 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":314,"threads_started":1}
I20260810 14:49:31.792398 15322 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9: No bootstrap required, opened a new log
I20260810 14:49:31.800249 15322 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b74c52694d64d7884569e86bab773b9" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39481 } }
I20260810 14:49:31.800648 15322 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:31.800786 15322 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9b74c52694d64d7884569e86bab773b9, State: Initialized, Role: FOLLOWER
I20260810 14:49:31.801362 15322 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b74c52694d64d7884569e86bab773b9" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39481 } }
I20260810 14:49:31.801599 15322 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:49:31.801730 15322 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:49:31.801908 15322 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:31.804275 15322 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b74c52694d64d7884569e86bab773b9" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39481 } }
I20260810 14:49:31.804776 15322 leader_election.cc:302] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9b74c52694d64d7884569e86bab773b9; no voters:
I20260810 14:49:31.805343 15322 leader_election.cc:288] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:49:31.805517 15328 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:31.806233 15328 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [term 1 LEADER]: Becoming Leader. State: Replica: 9b74c52694d64d7884569e86bab773b9, State: Running, Role: LEADER
I20260810 14:49:31.806748 15328 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b74c52694d64d7884569e86bab773b9" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39481 } }
I20260810 14:49:31.807505 15322 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:49:31.809734 15330 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9b74c52694d64d7884569e86bab773b9. Latest consensus state: current_term: 1 leader_uuid: "9b74c52694d64d7884569e86bab773b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b74c52694d64d7884569e86bab773b9" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39481 } } }
I20260810 14:49:31.810143 15330 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:31.815631 15335 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:49:31.814973 15329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9b74c52694d64d7884569e86bab773b9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9b74c52694d64d7884569e86bab773b9" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39481 } } }
I20260810 14:49:31.815904 15329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:31.820856 15335 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:49:31.830655 15335 catalog_manager.cc:1347] Generated new cluster ID: 973f1193872b41b48e4fd4a00c43bbd9
I20260810 14:49:31.830775 15335 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:49:31.844797 15335 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:49:31.846024 15335 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:49:31.854342 15335 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9: Generated new TSK 0
I20260810 14:49:31.855199 15335 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:49:31.865743 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:39481
--builtin_ntp_servers=127.24.215.148:41789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:49:32.220261 15351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:32.220527 15351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:32.220664 15351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:32.227891 15351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:32.228003 15351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:32.228204 15351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:49:32.237246 15351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:41789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:39481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:32.238549 15351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:32.240300 15351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:32.253521 15356 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:32.253607 15360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:32.253594 15357 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:32.256520 15351 server_base.cc:1034] running on GCE node
I20260810 14:49:32.257129 15351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:32.258004 15351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:32.259220 15351 hybrid_clock.cc:648] HybridClock initialized: now 1786373372259181 us; error 23 us; skew 500 ppm
I20260810 14:49:32.259580 15351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:49:32.261857 15351 webserver.cc:460] Webserver started at http://127.24.215.129:33943/ using document root <none> and password file <none>
I20260810 14:49:32.262542 15351 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:32.262666 15351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:32.262996 15351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:32.265884 15351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "e108a4a1b5ad4b34bdef8cd4e5b3a9ae"
format_stamp: "Formatted at 2026-08-10 14:49:32 on dist-test-slave-vkjp"
server_key: "b16e4b0042efdfb33b52a2581a3a3429"
server_key_iv: "099f15258d686b89f94a3ed7a416b812"
server_key_version: "encryptionkey@0"
I20260810 14:49:32.266669 15351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "e108a4a1b5ad4b34bdef8cd4e5b3a9ae"
format_stamp: "Formatted at 2026-08-10 14:49:32 on dist-test-slave-vkjp"
server_key: "b16e4b0042efdfb33b52a2581a3a3429"
server_key_iv: "099f15258d686b89f94a3ed7a416b812"
server_key_version: "encryptionkey@0"
I20260810 14:49:32.271833 15351 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260810 14:49:32.275234 15367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:32.276499 15351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:49:32.276674 15351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "e108a4a1b5ad4b34bdef8cd4e5b3a9ae"
format_stamp: "Formatted at 2026-08-10 14:49:32 on dist-test-slave-vkjp"
server_key: "b16e4b0042efdfb33b52a2581a3a3429"
server_key_iv: "099f15258d686b89f94a3ed7a416b812"
server_key_version: "encryptionkey@0"
I20260810 14:49:32.276895 15351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:32.296028 15351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:32.296826 15351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:32.297142 15351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:32.298101 15351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:32.299579 15351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:32.299717 15351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:32.299845 15351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:32.299933 15351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:32.333209 15351 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:32943
I20260810 14:49:32.333271 15495 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:32943 every 8 connection(s)
I20260810 14:49:32.334542 15351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:49:32.340080 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 15351
I20260810 14:49:32.340272 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:49:32.340857 25438 external_mini_cluster.cc:1428] Setting key 9b44612a68c5f5991178887230101e03
I20260810 14:49:32.343665 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:39481
--builtin_ntp_servers=127.24.215.148:41789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:49:32.347808 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:32.337101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:43913 (local address 127.24.215.190:39481)
0810 14:49:32.337556 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:32.337562 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:32.338726 (+ 1164us) server_negotiation.cc:407] Connection header received
0810 14:49:32.340580 (+ 1854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:32.340588 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:32.340678 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:32.340794 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:32.341992 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:32.342933 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:32.344621 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:32.345394 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:32.345542 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:32.346802 (+ 1260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:32.346811 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:32.346819 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:32.346843 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:32.346902 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:32.346920 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:32.347015 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:32.347258 (+ 243us) server_negotiation.cc:299] Negotiation successful
0810 14:49:32.347375 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":205,"threads_started":1}
I20260810 14:49:32.348493 15499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:32.337715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:39481 (local address 127.24.215.129:43913)
0810 14:49:32.338573 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:32.338607 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:32.340290 (+ 1683us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:32.341031 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:32.341039 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:32.341118 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:32.341824 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:32.341852 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:32.343076 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:32.343123 (+ 47us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:32.344449 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:32.344461 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:32.345598 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:32.345603 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:32.345800 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:32.346500 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:49:32.346502 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:32.346539 (+ 37us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:49:32.346616 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:32.347018 (+ 402us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:32.347026 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:32.347032 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:49:32.347217 (+ 185us) client_negotiation.cc:240] Negotiation successful
0810 14:49:32.347499 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":267,"threads_started":1}
I20260810 14:49:32.349905 15496 heartbeater.cc:347] Connected to a master server at 127.24.215.190:39481
I20260810 14:49:32.350207 15496 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:32.350925 15496 heartbeater.cc:511] Master 127.24.215.190:39481 requested a full tablet report, sending...
I20260810 14:49:32.352944 15281 ts_manager.cc:194] Registered new tserver with Master: e108a4a1b5ad4b34bdef8cd4e5b3a9ae (127.24.215.129:32943)
I20260810 14:49:32.354661 15281 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.215.129:43913
W20260810 14:49:32.684469 15500 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:32.684754 15500 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:32.684852 15500 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:32.692233 15500 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:32.692374 15500 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:32.692487 15500 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:49:32.701578 15500 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:41789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:39481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:32.703003 15500 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:32.704808 15500 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:32.718210 15506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:32.718233 15507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:32.718271 15509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:32.719218 15500 server_base.cc:1034] running on GCE node
I20260810 14:49:32.719812 15500 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:32.720783 15500 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:32.722067 15500 hybrid_clock.cc:648] HybridClock initialized: now 1786373372722014 us; error 40 us; skew 500 ppm
I20260810 14:49:32.722492 15500 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:49:32.725045 15500 webserver.cc:460] Webserver started at http://127.24.215.130:41377/ using document root <none> and password file <none>
I20260810 14:49:32.725811 15500 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:32.726032 15500 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:32.726403 15500 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:32.729502 15500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "80d34941cb8a43dcba910454775c5ccd"
format_stamp: "Formatted at 2026-08-10 14:49:32 on dist-test-slave-vkjp"
server_key: "cba8163b9b9830a3a053bb05e0cf85f9"
server_key_iv: "d9125b59368ae83389084d1aa006ca94"
server_key_version: "encryptionkey@0"
I20260810 14:49:32.730343 15500 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "80d34941cb8a43dcba910454775c5ccd"
format_stamp: "Formatted at 2026-08-10 14:49:32 on dist-test-slave-vkjp"
server_key: "cba8163b9b9830a3a053bb05e0cf85f9"
server_key_iv: "d9125b59368ae83389084d1aa006ca94"
server_key_version: "encryptionkey@0"
I20260810 14:49:32.735805 15500 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 14:49:32.739311 15516 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:32.740583 15500 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:49:32.740761 15500 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "80d34941cb8a43dcba910454775c5ccd"
format_stamp: "Formatted at 2026-08-10 14:49:32 on dist-test-slave-vkjp"
server_key: "cba8163b9b9830a3a053bb05e0cf85f9"
server_key_iv: "d9125b59368ae83389084d1aa006ca94"
server_key_version: "encryptionkey@0"
I20260810 14:49:32.740984 15500 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:32.760175 15500 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:32.761005 15500 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:32.761348 15500 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:32.762331 15500 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:32.763914 15500 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:32.764047 15500 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:32.764173 15500 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:32.764278 15500 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:32.806037 15500 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:43143
I20260810 14:49:32.806044 15646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:43143 every 8 connection(s)
I20260810 14:49:32.807416 15500 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:49:32.814687 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 15500
I20260810 14:49:32.814888 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:49:32.815507 25438 external_mini_cluster.cc:1428] Setting key e1823c11b1b21a898a79912fcae5afd3
I20260810 14:49:32.818663 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:39481
--builtin_ntp_servers=127.24.215.148:41789
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:49:32.820571 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:32.810102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:41699 (local address 127.24.215.190:39481)
0810 14:49:32.810329 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:32.810333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:32.811687 (+ 1354us) server_negotiation.cc:407] Connection header received
0810 14:49:32.813399 (+ 1712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:32.813404 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:32.813474 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:32.813569 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:32.814679 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:32.815811 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:32.817680 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:32.818100 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:32.818241 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:32.819724 (+ 1483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:32.819729 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:32.819732 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:32.819750 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:32.819783 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:32.819803 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:32.819899 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:32.820169 (+ 270us) server_negotiation.cc:299] Negotiation successful
0810 14:49:32.820269 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 14:49:32.821692 15650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:32.810545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:39481 (local address 127.24.215.130:41699)
0810 14:49:32.811521 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:32.811565 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:32.813149 (+ 1584us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:32.813778 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:32.813787 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:32.813882 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:32.814486 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:32.814503 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:32.815975 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:32.815979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:32.817531 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:32.817544 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:32.818344 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:32.818352 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:32.818584 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:32.819386 (+ 802us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:49:32.819389 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:32.819431 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:49:32.819513 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:32.819916 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:32.819943 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:32.819949 (+ 6us) client_negotiation.cc:769] Sending connection context
0810 14:49:32.820130 (+ 181us) client_negotiation.cc:240] Negotiation successful
0810 14:49:32.820544 (+ 414us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":278,"threads_started":1}
I20260810 14:49:32.823170 15648 heartbeater.cc:347] Connected to a master server at 127.24.215.190:39481
I20260810 14:49:32.823551 15648 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:32.824529 15648 heartbeater.cc:511] Master 127.24.215.190:39481 requested a full tablet report, sending...
I20260810 14:49:32.826385 15281 ts_manager.cc:194] Registered new tserver with Master: 80d34941cb8a43dcba910454775c5ccd (127.24.215.130:43143)
I20260810 14:49:32.827772 15281 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.215.130:41699
W20260810 14:49:33.154217 15651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:33.154495 15651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:33.154627 15651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:33.161861 15651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:33.161950 15651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:33.162130 15651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:49:33.171074 15651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:41789
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:39481
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:33.172405 15651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:33.174232 15651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:33.187623 15657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:33.187594 15658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:33.187530 15660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:33.188694 15651 server_base.cc:1034] running on GCE node
I20260810 14:49:33.189330 15651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:33.190266 15651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:33.191483 15651 hybrid_clock.cc:648] HybridClock initialized: now 1786373373191413 us; error 58 us; skew 500 ppm
I20260810 14:49:33.191879 15651 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:49:33.194320 15651 webserver.cc:460] Webserver started at http://127.24.215.131:38185/ using document root <none> and password file <none>
I20260810 14:49:33.195014 15651 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:33.195190 15651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:33.195518 15651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:33.198417 15651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "b8b395c4817a429bae9e5d62784abc86"
format_stamp: "Formatted at 2026-08-10 14:49:33 on dist-test-slave-vkjp"
server_key: "b34ac2640263e4b55a5dd30ada20d121"
server_key_iv: "3723ee7ef0ea69be9da791df3ebd63a2"
server_key_version: "encryptionkey@0"
I20260810 14:49:33.199281 15651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "b8b395c4817a429bae9e5d62784abc86"
format_stamp: "Formatted at 2026-08-10 14:49:33 on dist-test-slave-vkjp"
server_key: "b34ac2640263e4b55a5dd30ada20d121"
server_key_iv: "3723ee7ef0ea69be9da791df3ebd63a2"
server_key_version: "encryptionkey@0"
I20260810 14:49:33.204437 15651 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:49:33.207942 15668 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:33.209256 15651 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:49:33.209434 15651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "b8b395c4817a429bae9e5d62784abc86"
format_stamp: "Formatted at 2026-08-10 14:49:33 on dist-test-slave-vkjp"
server_key: "b34ac2640263e4b55a5dd30ada20d121"
server_key_iv: "3723ee7ef0ea69be9da791df3ebd63a2"
server_key_version: "encryptionkey@0"
I20260810 14:49:33.209648 15651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:33.230351 15651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:33.231174 15651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:33.231495 15651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:33.232429 15651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:33.233912 15651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:33.234033 15651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:33.234166 15651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:33.234252 15651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:33.269117 15651 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:34121
I20260810 14:49:33.269202 15796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:34121 every 8 connection(s)
I20260810 14:49:33.270447 15651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:49:33.280496 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 15651
I20260810 14:49:33.280704 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:49:33.281337 25438 external_mini_cluster.cc:1428] Setting key 9960e84e2849ce9f7077f920f00afb0b
I20260810 14:49:33.283385 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.273056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:51485 (local address 127.24.215.190:39481)
0810 14:49:33.273247 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:33.273252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:33.274577 (+ 1325us) server_negotiation.cc:407] Connection header received
0810 14:49:33.276451 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:33.276456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:33.276547 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:33.276662 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:33.277700 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.278462 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.280100 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.280585 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.280721 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:33.282470 (+ 1749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:33.282479 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:33.282485 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:33.282506 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:33.282548 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:33.282563 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:33.282686 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:33.282900 (+ 214us) server_negotiation.cc:299] Negotiation successful
0810 14:49:33.283007 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 14:49:33.284642 15799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.273564 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:39481 (local address 127.24.215.131:51485)
0810 14:49:33.274415 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:33.274449 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:33.276187 (+ 1738us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:33.276886 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:33.276894 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:33.276981 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:33.277534 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:33.277550 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.278628 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.278632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:33.279922 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:33.279934 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.280865 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.280900 (+ 35us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:33.281244 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:33.282064 (+ 820us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:49:33.282068 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:33.282123 (+ 55us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:49:33.282226 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:33.282686 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:33.282698 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:33.282708 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:49:33.282886 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 14:49:33.283406 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":268,"threads_started":1}
I20260810 14:49:33.285835 15797 heartbeater.cc:347] Connected to a master server at 127.24.215.190:39481
I20260810 14:49:33.286131 15797 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:33.286917 15797 heartbeater.cc:511] Master 127.24.215.190:39481 requested a full tablet report, sending...
I20260810 14:49:33.288283 15281 ts_manager.cc:194] Registered new tserver with Master: b8b395c4817a429bae9e5d62784abc86 (127.24.215.131:34121)
I20260810 14:49:33.289184 15281 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.215.131:51485
I20260810 14:49:33.296651 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:49:33.307003 15498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.300982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56318 (local address 127.24.215.190:39481)
0810 14:49:33.301199 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:33.301204 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:33.301578 (+ 374us) server_negotiation.cc:407] Connection header received
0810 14:49:33.301748 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:33.301752 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:33.301826 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:33.301908 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:33.302467 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.303246 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.304596 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.305161 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.305255 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:33.306329 (+ 1074us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:33.306334 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:33.306337 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:33.306354 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:33.306385 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:33.306397 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:33.306519 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:33.306732 (+ 213us) server_negotiation.cc:299] Negotiation successful
0810 14:49:33.306775 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 14:49:33.311182 15281 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56318:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:49:33.318390 15281 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:49:33.355883 15812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.330757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:32943 (local address 127.0.0.1:39630)
0810 14:49:33.343647 (+ 12890us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:33.343698 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:33.343924 (+ 226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:33.348418 (+ 4494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:33.348425 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:33.348449 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:33.348955 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:33.348973 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.350840 (+ 1867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.350847 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:33.353029 (+ 2182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:33.353046 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.354914 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.354921 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:33.355148 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:33.355204 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 14:49:33.355337 (+ 133us) client_negotiation.cc:240] Negotiation successful
0810 14:49:33.355538 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12564,"thread_start_us":13358,"threads_started":1}
I20260810 14:49:33.356267 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.330866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39630 (local address 127.24.215.129:32943)
0810 14:49:33.347779 (+ 16913us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:33.347785 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:33.347809 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:49:33.347888 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:33.347893 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:33.348171 (+ 278us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:33.349194 (+ 1023us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:33.349406 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.350696 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.353243 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.354722 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.354938 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:33.355174 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:33.355568 (+ 394us) server_negotiation.cc:299] Negotiation successful
0810 14:49:33.355732 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16758,"thread_start_us":16656,"threads_started":1}
I20260810 14:49:33.357055 15814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.331117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49230 (local address 127.24.215.130:43143)
0810 14:49:33.346354 (+ 15237us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:33.346365 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:33.348284 (+ 1919us) server_negotiation.cc:407] Connection header received
0810 14:49:33.348605 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:33.348614 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:33.348830 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:33.348992 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:33.349955 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.351551 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.353933 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.355058 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.355246 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:33.355437 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:33.356088 (+ 651us) server_negotiation.cc:299] Negotiation successful
0810 14:49:33.356699 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15013,"thread_start_us":15245,"threads_started":1}
I20260810 14:49:33.357427 15815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.330778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:43143 (local address 127.0.0.1:49230)
0810 14:49:33.348150 (+ 17372us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:33.348169 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:33.348414 (+ 245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:33.349325 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:33.349329 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:33.349354 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:33.349759 (+ 405us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:33.349769 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.351692 (+ 1923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.351703 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:33.353751 (+ 2048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:33.353763 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.355331 (+ 1568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.355337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:33.355529 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:33.355558 (+ 29us) client_negotiation.cc:769] Sending connection context
0810 14:49:33.355644 (+ 86us) client_negotiation.cc:240] Negotiation successful
0810 14:49:33.355755 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17211,"mutex_wait_us":44,"thread_start_us":17014,"threads_started":1}
I20260810 14:49:33.357618 15496 heartbeater.cc:503] Master 127.24.215.190:39481 was elected leader, sending a full tablet report...
I20260810 14:49:33.358361 15811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.333935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:34121 (local address 127.0.0.1:54006)
0810 14:49:33.348204 (+ 14269us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:33.348225 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:33.348432 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:33.349402 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:33.349406 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:33.349434 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:33.349950 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:33.349964 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.352068 (+ 2104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.352074 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:33.354409 (+ 2335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:33.354425 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.356560 (+ 2135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.356567 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:33.356744 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:33.356775 (+ 31us) client_negotiation.cc:769] Sending connection context
0810 14:49:33.356864 (+ 89us) client_negotiation.cc:240] Negotiation successful
0810 14:49:33.356978 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14117,"mutex_wait_us":25,"thread_start_us":13503,"threads_started":1}
I20260810 14:49:33.358500 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.334712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54006 (local address 127.24.215.131:34121)
0810 14:49:33.347230 (+ 12518us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:33.347250 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:33.348336 (+ 1086us) server_negotiation.cc:407] Connection header received
0810 14:49:33.348641 (+ 305us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:33.348648 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:33.348871 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:33.349032 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:33.350144 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.351722 (+ 1578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.354626 (+ 2904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.356026 (+ 1400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.356200 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:33.356419 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:33.357947 (+ 1528us) server_negotiation.cc:299] Negotiation successful
0810 14:49:33.358130 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12271,"thread_start_us":12121,"threads_started":1}
I20260810 14:49:33.360622 15419 tablet_service.cc:1467] Processing CreateTablet for tablet 5706210999eb497c8ce20748fec35bbe (DEFAULT_TABLE table=test-table [id=59b161227bcb42fd94f976d63029b2ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:33.360813 15571 tablet_service.cc:1467] Processing CreateTablet for tablet 5706210999eb497c8ce20748fec35bbe (DEFAULT_TABLE table=test-table [id=59b161227bcb42fd94f976d63029b2ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:33.362156 15419 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5706210999eb497c8ce20748fec35bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:33.362402 15571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5706210999eb497c8ce20748fec35bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:33.362509 15723 tablet_service.cc:1467] Processing CreateTablet for tablet 5706210999eb497c8ce20748fec35bbe (DEFAULT_TABLE table=test-table [id=59b161227bcb42fd94f976d63029b2ea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:33.363883 15723 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5706210999eb497c8ce20748fec35bbe. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:33.373360 15817 tablet_bootstrap.cc:492] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd: Bootstrap starting.
I20260810 14:49:33.377233 15817 tablet_bootstrap.cc:654] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:33.378134 15819 tablet_bootstrap.cc:492] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86: Bootstrap starting.
I20260810 14:49:33.378618 15817 log.cc:824] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:33.380350 15818 tablet_bootstrap.cc:492] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae: Bootstrap starting.
I20260810 14:49:33.382198 15819 tablet_bootstrap.cc:654] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:33.382776 15817 tablet_bootstrap.cc:492] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd: No bootstrap required, opened a new log
I20260810 14:49:33.383118 15817 ts_tablet_manager.cc:1397] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.007s
I20260810 14:49:33.383555 15819 log.cc:824] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:33.383998 15818 tablet_bootstrap.cc:654] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:33.385456 15818 log.cc:824] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:33.388327 15818 tablet_bootstrap.cc:492] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae: No bootstrap required, opened a new log
I20260810 14:49:33.388607 15818 ts_tablet_manager.cc:1397] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260810 14:49:33.388830 15819 tablet_bootstrap.cc:492] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86: No bootstrap required, opened a new log
I20260810 14:49:33.389101 15819 ts_tablet_manager.cc:1397] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86: Time spent bootstrapping tablet: real 0.011s user 0.009s sys 0.000s
I20260810 14:49:33.393234 15817 raft_consensus.cc:348] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e108a4a1b5ad4b34bdef8cd4e5b3a9ae" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32943 } } peers { permanent_uuid: "80d34941cb8a43dcba910454775c5ccd" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 43143 } } peers { permanent_uuid: "b8b395c4817a429bae9e5d62784abc86" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34121 } }
I20260810 14:49:33.393697 15817 raft_consensus.cc:374] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:33.393792 15817 raft_consensus.cc:729] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80d34941cb8a43dcba910454775c5ccd, State: Initialized, Role: FOLLOWER
I20260810 14:49:33.394492 15817 consensus_queue.cc:260] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e108a4a1b5ad4b34bdef8cd4e5b3a9ae" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32943 } } peers { permanent_uuid: "80d34941cb8a43dcba910454775c5ccd" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 43143 } } peers { permanent_uuid: "b8b395c4817a429bae9e5d62784abc86" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34121 } }
I20260810 14:49:33.396306 15648 heartbeater.cc:503] Master 127.24.215.190:39481 was elected leader, sending a full tablet report...
I20260810 14:49:33.396476 15818 raft_consensus.cc:348] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e108a4a1b5ad4b34bdef8cd4e5b3a9ae" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32943 } } peers { permanent_uuid: "80d34941cb8a43dcba910454775c5ccd" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 43143 } } peers { permanent_uuid: "b8b395c4817a429bae9e5d62784abc86" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34121 } }
I20260810 14:49:33.396898 15818 raft_consensus.cc:374] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:33.396924 15817 ts_tablet_manager.cc:1428] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260810 14:49:33.396984 15818 raft_consensus.cc:729] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e108a4a1b5ad4b34bdef8cd4e5b3a9ae, State: Initialized, Role: FOLLOWER
I20260810 14:49:33.397643 15818 consensus_queue.cc:260] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e108a4a1b5ad4b34bdef8cd4e5b3a9ae" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32943 } } peers { permanent_uuid: "80d34941cb8a43dcba910454775c5ccd" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 43143 } } peers { permanent_uuid: "b8b395c4817a429bae9e5d62784abc86" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34121 } }
I20260810 14:49:33.399883 15818 ts_tablet_manager.cc:1428] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260810 14:49:33.399870 15819 raft_consensus.cc:348] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e108a4a1b5ad4b34bdef8cd4e5b3a9ae" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32943 } } peers { permanent_uuid: "80d34941cb8a43dcba910454775c5ccd" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 43143 } } peers { permanent_uuid: "b8b395c4817a429bae9e5d62784abc86" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34121 } }
I20260810 14:49:33.400279 15819 raft_consensus.cc:374] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:33.400408 15819 raft_consensus.cc:729] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8b395c4817a429bae9e5d62784abc86, State: Initialized, Role: FOLLOWER
I20260810 14:49:33.400976 15819 consensus_queue.cc:260] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e108a4a1b5ad4b34bdef8cd4e5b3a9ae" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32943 } } peers { permanent_uuid: "80d34941cb8a43dcba910454775c5ccd" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 43143 } } peers { permanent_uuid: "b8b395c4817a429bae9e5d62784abc86" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34121 } }
I20260810 14:49:33.402383 15819 ts_tablet_manager.cc:1428] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86: Time spent starting tablet: real 0.013s user 0.005s sys 0.009s
I20260810 14:49:33.402745 15797 heartbeater.cc:503] Master 127.24.215.190:39481 was elected leader, sending a full tablet report...
W20260810 14:49:33.522222 15798 tablet.cc:2369] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:49:33.559680 15649 tablet.cc:2369] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:49:33.587131 15497 tablet.cc:2369] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:49:33.630497 15823 raft_consensus.cc:482] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:49:33.630882 15823 raft_consensus.cc:504] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e108a4a1b5ad4b34bdef8cd4e5b3a9ae" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32943 } } peers { permanent_uuid: "80d34941cb8a43dcba910454775c5ccd" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 43143 } } peers { permanent_uuid: "b8b395c4817a429bae9e5d62784abc86" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34121 } }
I20260810 14:49:33.632735 15823 leader_election.cc:288] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e108a4a1b5ad4b34bdef8cd4e5b3a9ae (127.24.215.129:32943), b8b395c4817a429bae9e5d62784abc86 (127.24.215.131:34121)
I20260810 14:49:33.641100 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.633408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:33495 (local address 127.24.215.129:32943)
0810 14:49:33.633658 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:33.633667 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:33.634292 (+ 625us) server_negotiation.cc:407] Connection header received
0810 14:49:33.634526 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:33.634531 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:33.634617 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:33.634802 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:33.635687 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.636486 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.638882 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.640036 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.640231 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:33.640331 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:33.640722 (+ 391us) server_negotiation.cc:299] Negotiation successful
0810 14:49:33.640861 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99}
I20260810 14:49:33.641120 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.633533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:54867 (local address 127.24.215.131:34121)
0810 14:49:33.633860 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:33.633867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:33.634576 (+ 709us) server_negotiation.cc:407] Connection header received
0810 14:49:33.634766 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:33.634771 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:33.634871 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:33.634980 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:33.635731 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.636524 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.638753 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.639843 (+ 1090us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.640006 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:33.640101 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:33.640676 (+ 575us) server_negotiation.cc:299] Negotiation successful
0810 14:49:33.640818 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260810 14:49:33.641165 15826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.633331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:34121 (local address 127.24.215.130:54867)
0810 14:49:33.634460 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:33.634478 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:33.634619 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:33.635001 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:33.635006 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:33.635046 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:33.635541 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:33.635552 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.636661 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.636668 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:33.638559 (+ 1891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:33.638572 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.640283 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.640292 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:33.640477 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:33.640522 (+ 45us) client_negotiation.cc:769] Sending connection context
0810 14:49:33.640599 (+ 77us) client_negotiation.cc:240] Negotiation successful
0810 14:49:33.640754 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":999,"mutex_wait_us":107,"thread_start_us":435,"threads_started":1}
I20260810 14:49:33.641280 15827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.633241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:32943 (local address 127.24.215.130:33495)
0810 14:49:33.634133 (+ 892us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:33.634158 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:33.634357 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:33.634859 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:33.634864 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:33.634905 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:33.635495 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:33.635509 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.636641 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.636648 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:33.638579 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:33.638595 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.640302 (+ 1707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.640310 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:33.640522 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:33.640549 (+ 27us) client_negotiation.cc:769] Sending connection context
0810 14:49:33.640679 (+ 130us) client_negotiation.cc:240] Negotiation successful
0810 14:49:33.640827 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":156,"thread_start_us":795,"threads_started":1}
I20260810 14:49:33.642551 15443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5706210999eb497c8ce20748fec35bbe" candidate_uuid: "80d34941cb8a43dcba910454775c5ccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e108a4a1b5ad4b34bdef8cd4e5b3a9ae" is_pre_election: true
I20260810 14:49:33.642556 15746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5706210999eb497c8ce20748fec35bbe" candidate_uuid: "80d34941cb8a43dcba910454775c5ccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8b395c4817a429bae9e5d62784abc86" is_pre_election: true
I20260810 14:49:33.643117 15746 raft_consensus.cc:2454] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80d34941cb8a43dcba910454775c5ccd in term 0.
I20260810 14:49:33.643126 15443 raft_consensus.cc:2454] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 80d34941cb8a43dcba910454775c5ccd in term 0.
I20260810 14:49:33.643821 15519 leader_election.cc:302] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80d34941cb8a43dcba910454775c5ccd, e108a4a1b5ad4b34bdef8cd4e5b3a9ae; no voters:
I20260810 14:49:33.644238 15823 raft_consensus.cc:2781] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:49:33.644394 15823 raft_consensus.cc:482] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:49:33.644500 15823 raft_consensus.cc:3037] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:33.646829 15823 raft_consensus.cc:504] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e108a4a1b5ad4b34bdef8cd4e5b3a9ae" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32943 } } peers { permanent_uuid: "80d34941cb8a43dcba910454775c5ccd" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 43143 } } peers { permanent_uuid: "b8b395c4817a429bae9e5d62784abc86" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34121 } }
I20260810 14:49:33.647513 15823 leader_election.cc:288] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [CANDIDATE]: Term 1 election: Requested vote from peers e108a4a1b5ad4b34bdef8cd4e5b3a9ae (127.24.215.129:32943), b8b395c4817a429bae9e5d62784abc86 (127.24.215.131:34121)
I20260810 14:49:33.647998 15443 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5706210999eb497c8ce20748fec35bbe" candidate_uuid: "80d34941cb8a43dcba910454775c5ccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e108a4a1b5ad4b34bdef8cd4e5b3a9ae"
I20260810 14:49:33.648213 15443 raft_consensus.cc:3037] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:33.648244 15746 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5706210999eb497c8ce20748fec35bbe" candidate_uuid: "80d34941cb8a43dcba910454775c5ccd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8b395c4817a429bae9e5d62784abc86"
I20260810 14:49:33.648509 15746 raft_consensus.cc:3037] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:33.650681 15746 raft_consensus.cc:2454] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80d34941cb8a43dcba910454775c5ccd in term 1.
I20260810 14:49:33.650703 15443 raft_consensus.cc:2454] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 80d34941cb8a43dcba910454775c5ccd in term 1.
I20260810 14:49:33.651130 15520 leader_election.cc:302] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 80d34941cb8a43dcba910454775c5ccd, b8b395c4817a429bae9e5d62784abc86; no voters:
I20260810 14:49:33.651454 15823 raft_consensus.cc:2781] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:33.651948 15823 raft_consensus.cc:686] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [term 1 LEADER]: Becoming Leader. State: Replica: 80d34941cb8a43dcba910454775c5ccd, State: Running, Role: LEADER
I20260810 14:49:33.652426 15823 consensus_queue.cc:237] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e108a4a1b5ad4b34bdef8cd4e5b3a9ae" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32943 } } peers { permanent_uuid: "80d34941cb8a43dcba910454775c5ccd" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 43143 } } peers { permanent_uuid: "b8b395c4817a429bae9e5d62784abc86" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34121 } }
I20260810 14:49:33.657390 15279 catalog_manager.cc:5495] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd reported cstate change: term changed from 0 to 1, leader changed from <none> to 80d34941cb8a43dcba910454775c5ccd (127.24.215.130). New cstate: current_term: 1 leader_uuid: "80d34941cb8a43dcba910454775c5ccd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e108a4a1b5ad4b34bdef8cd4e5b3a9ae" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32943 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "80d34941cb8a43dcba910454775c5ccd" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 43143 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b8b395c4817a429bae9e5d62784abc86" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34121 } health_report { overall_health: UNKNOWN } } }
I20260810 14:49:33.725320 15814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.719043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49238 (local address 127.24.215.130:43143)
0810 14:49:33.719283 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:33.719291 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:33.719312 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:49:33.719556 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:33.719560 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:33.719659 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:33.719813 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:49:33.720296 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.721067 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.722540 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.722970 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.723122 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:33.724018 (+ 896us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:49:33.724609 (+ 591us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:49:33.724820 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:33.724992 (+ 172us) server_negotiation.cc:299] Negotiation successful
0810 14:49:33.725040 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 14:49:33.734480 15443 raft_consensus.cc:1261] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae [term 1 FOLLOWER]: Refusing update from remote peer 80d34941cb8a43dcba910454775c5ccd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:49:33.734521 15746 raft_consensus.cc:1261] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86 [term 1 FOLLOWER]: Refusing update from remote peer 80d34941cb8a43dcba910454775c5ccd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:49:33.735559 15823 consensus_queue.cc:1035] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [LEADER]: Connected to new peer: Peer: permanent_uuid: "e108a4a1b5ad4b34bdef8cd4e5b3a9ae" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 32943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:49:33.735862 15828 consensus_queue.cc:1035] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8b395c4817a429bae9e5d62784abc86" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 34121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:49:33.747260 15832 mvcc.cc:205] Tried to move back new op lower bound from 7316985338805178368 to 7316985338484420608. Current Snapshot: MvccSnapshot[applied={T|T < 7316985338805178368}]
I20260810 14:49:33.750187 15833 mvcc.cc:205] Tried to move back new op lower bound from 7316985338805178368 to 7316985338484420608. Current Snapshot: MvccSnapshot[applied={T|T < 7316985338805178368}]
I20260810 14:49:33.764014 15281 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:49:33.764355 15281 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56318:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:49:33.768707 15281 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9b74c52694d64d7884569e86bab773b9: Sending DeleteTablet for 3 replicas of tablet 5706210999eb497c8ce20748fec35bbe
I20260810 14:49:33.770210 15419 tablet_service.cc:1514] Processing DeleteTablet for tablet 5706210999eb497c8ce20748fec35bbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:33 UTC) from {username='slave'} at 127.0.0.1:39630
I20260810 14:49:33.770378 15571 tablet_service.cc:1514] Processing DeleteTablet for tablet 5706210999eb497c8ce20748fec35bbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:33 UTC) from {username='slave'} at 127.0.0.1:49230
I20260810 14:49:33.771399 15723 tablet_service.cc:1514] Processing DeleteTablet for tablet 5706210999eb497c8ce20748fec35bbe with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:33 UTC) from {username='slave'} at 127.0.0.1:54006
I20260810 14:49:33.773196 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 15351
I20260810 14:49:33.786849 15844 tablet_replica.cc:330] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd: stopping tablet replica
I20260810 14:49:33.788091 15844 raft_consensus.cc:2229] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:49:33.791551 15844 raft_consensus.cc:2258] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:33.791949 15845 tablet_replica.cc:330] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86: stopping tablet replica
I20260810 14:49:33.792476 15843 tablet_replica.cc:330] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae: stopping tablet replica
I20260810 14:49:33.792439 15845 raft_consensus.cc:2229] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:49:33.792897 15843 raft_consensus.cc:2229] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:49:33.793036 15845 raft_consensus.cc:2258] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:33.793401 15843 raft_consensus.cc:2258] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:33.794785 15844 ts_tablet_manager.cc:1905] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:33.796288 15843 ts_tablet_manager.cc:1905] T 5706210999eb497c8ce20748fec35bbe P e108a4a1b5ad4b34bdef8cd4e5b3a9ae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:33.796281 15845 ts_tablet_manager.cc:1905] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:33.797688 15813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.775769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39644 (local address 127.24.215.129:32943)
0810 14:49:33.775973 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:33.775981 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:33.779433 (+ 3452us) server_negotiation.cc:407] Connection header received
0810 14:49:33.780366 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:33.780372 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:33.780489 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:33.780578 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:33.782012 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.783246 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.788573 (+ 5327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.789298 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.789427 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:33.795935 (+ 6508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:33.795946 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:33.795963 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:33.796690 (+ 727us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:33.796757 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:33.796782 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:33.796892 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:33.797186 (+ 294us) server_negotiation.cc:299] Negotiation successful
0810 14:49:33.797312 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 14:49:33.799052 15491 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:33.805421 15845 ts_tablet_manager.cc:1918] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:49:33.805622 15845 log.cc:1196] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/5706210999eb497c8ce20748fec35bbe
I20260810 14:49:33.806185 15845 ts_tablet_manager.cc:1939] T 5706210999eb497c8ce20748fec35bbe P b8b395c4817a429bae9e5d62784abc86: Deleting consensus metadata
I20260810 14:49:33.808555 15267 catalog_manager.cc:4841] TS b8b395c4817a429bae9e5d62784abc86 (127.24.215.131:34121): tablet 5706210999eb497c8ce20748fec35bbe (table test-table [id=59b161227bcb42fd94f976d63029b2ea]) successfully deleted
I20260810 14:49:33.816668 15844 ts_tablet_manager.cc:1918] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:49:33.816962 15844 log.cc:1196] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/5706210999eb497c8ce20748fec35bbe
I20260810 14:49:33.817685 15844 ts_tablet_manager.cc:1939] T 5706210999eb497c8ce20748fec35bbe P 80d34941cb8a43dcba910454775c5ccd: Deleting consensus metadata
I20260810 14:49:33.819824 15264 catalog_manager.cc:4841] TS 80d34941cb8a43dcba910454775c5ccd (127.24.215.130:43143): tablet 5706210999eb497c8ce20748fec35bbe (table test-table [id=59b161227bcb42fd94f976d63029b2ea]) successfully deleted
I20260810 14:49:33.904366 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 15351
W20260810 14:49:33.926610 15266 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.215.129:32943 (error 108)
I20260810 14:49:33.926867 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 15500
I20260810 14:49:33.928936 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.928144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:32943 (local address 127.0.0.1:39658)
0810 14:49:33.928605 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:33.928756 (+ 151us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:32943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":235,"threads_started":1}
W20260810 14:49:33.929255 15266 catalog_manager.cc:4584] TS e108a4a1b5ad4b34bdef8cd4e5b3a9ae (127.24.215.129:32943): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5706210999eb497c8ce20748fec35bbe: Network error: Client connection negotiation failed: client connection to 127.24.215.129:32943: connect: Connection refused (error 111)
I20260810 14:49:33.935555 15814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.927854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49242 (local address 127.24.215.130:43143)
0810 14:49:33.928054 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:33.928063 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:33.928084 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:49:33.928301 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:33.928309 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:33.928436 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:33.928549 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:33.929151 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.930368 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.931777 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:33.932514 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:33.932653 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:33.933775 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:33.933791 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:33.933810 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:33.934667 (+ 857us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:33.934735 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:33.934760 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:33.934889 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:33.935019 (+ 130us) server_negotiation.cc:299] Negotiation successful
0810 14:49:33.935186 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 14:49:33.936422 15641 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:33.983448 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.982809 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:32943 (local address 127.0.0.1:39670)
0810 14:49:33.983126 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:33.983242 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:32943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260810 14:49:33.984836 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:33.984360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:32943 (local address 127.0.0.1:39684)
0810 14:49:33.984571 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:33.984661 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:32943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260810 14:49:34.045820 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:34.045225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:32943 (local address 127.0.0.1:39692)
0810 14:49:34.045486 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:34.045604 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:32943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
I20260810 14:49:34.046916 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:34.046478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:32943 (local address 127.0.0.1:39702)
0810 14:49:34.046647 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:34.046730 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:32943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260810 14:49:34.052584 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 15500
I20260810 14:49:34.076395 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 15651
I20260810 14:49:34.082924 15816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:34.077195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54008 (local address 127.24.215.131:34121)
0810 14:49:34.077385 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:34.077390 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:34.077409 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:49:34.077577 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:34.077581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:34.077665 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:34.077758 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:34.078105 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:34.078823 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:34.080136 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:34.080535 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:34.080626 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:34.081597 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:34.081606 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:34.081620 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:34.082154 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:34.082199 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:34.082218 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:34.082310 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:34.082444 (+ 134us) server_negotiation.cc:299] Negotiation successful
0810 14:49:34.082538 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 14:49:34.083693 15791 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:34.127470 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:34.126927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:32943 (local address 127.0.0.1:39718)
0810 14:49:34.127201 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:34.127304 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:32943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260810 14:49:34.128669 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:34.128207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:32943 (local address 127.0.0.1:39724)
0810 14:49:34.128462 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:34.128535 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:32943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
I20260810 14:49:34.172429 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 15651
I20260810 14:49:34.193388 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 15247
I20260810 14:49:34.194437 15316 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:34.304881 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:34.304259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:32943 (local address 127.0.0.1:39728)
0810 14:49:34.304572 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:34.304697 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:32943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135}
I20260810 14:49:34.304878 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 15247
2026-08-10T14:49:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (3065 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:49:34 dist-test-slave-vkjp krb5kdc[15858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:49:34 dist-test-slave-vkjp krb5kdc[15858](info): set up 2 sockets
Aug 10 14:49:34 dist-test-slave-vkjp krb5kdc[15858](info): commencing operation
krb5kdc: starting...
W20260810 14:49:35.519440 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:49:35 dist-test-slave-vkjp krb5kdc[15858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:49:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:49:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:49:35.826632 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:39031
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:39031
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:36.170900 15871 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:36.171216 15871 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:36.171404 15871 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:36.178673 15871 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:49:36.178774 15871 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:36.178885 15871 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:36.178980 15871 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:49:36.179065 15871 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:49:36.188889 15871 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:39031
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:39031
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:36.190280 15871 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:36.191727 15871 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:36.203727 15877 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:36.203799 15878 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:36.203771 15880 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:36.204895 15871 server_base.cc:1034] running on GCE node
I20260810 14:49:36.205750 15871 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:36.207302 15871 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:36.208585 15871 hybrid_clock.cc:648] HybridClock initialized: now 1786373376208520 us; error 45 us; skew 500 ppm
I20260810 14:49:36.209025 15871 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:36 dist-test-slave-vkjp krb5kdc[15858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373376, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:36.212384 15871 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:49:36.213976 15871 webserver.cc:460] Webserver started at http://127.24.215.190:36393/ using document root <none> and password file <none>
I20260810 14:49:36.214696 15871 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:36.214846 15871 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:36.215258 15871 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:36.218319 15871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "f597ea5d9c5840d0be54308b286a76d7"
format_stamp: "Formatted at 2026-08-10 14:49:36 on dist-test-slave-vkjp"
server_key: "0cc96b40be2457c883d0cc2bf47180ac"
server_key_iv: "2f7d72080c65fbd794e4569deedb2362"
server_key_version: "encryptionkey@0"
I20260810 14:49:36.219193 15871 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "f597ea5d9c5840d0be54308b286a76d7"
format_stamp: "Formatted at 2026-08-10 14:49:36 on dist-test-slave-vkjp"
server_key: "0cc96b40be2457c883d0cc2bf47180ac"
server_key_iv: "2f7d72080c65fbd794e4569deedb2362"
server_key_version: "encryptionkey@0"
I20260810 14:49:36.224630 15871 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 14:49:36.228252 15887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:36.229635 15871 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 14:49:36.229833 15871 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "f597ea5d9c5840d0be54308b286a76d7"
format_stamp: "Formatted at 2026-08-10 14:49:36 on dist-test-slave-vkjp"
server_key: "0cc96b40be2457c883d0cc2bf47180ac"
server_key_iv: "2f7d72080c65fbd794e4569deedb2362"
server_key_version: "encryptionkey@0"
I20260810 14:49:36.230059 15871 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:36.263037 15871 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:36.267593 15871 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:36.267892 15871 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:36.287734 15871 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:39031
I20260810 14:49:36.287767 15950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:39031 every 8 connection(s)
I20260810 14:49:36.289112 15871 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:49:36.292986 15951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:36.297176 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 15871
I20260810 14:49:36.297381 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:49:36.297828 25438 external_mini_cluster.cc:1428] Setting key 26e3416a940e7de2a9fae601de5baa86
I20260810 14:49:36.305634 15951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7: Bootstrap starting.
I20260810 14:49:36.309852 15951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:36.311265 15951 log.cc:824] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:36.314366 15951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7: No bootstrap required, opened a new log
Aug 10 14:49:36 dist-test-slave-vkjp krb5kdc[15858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:49:36.324755 15954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:36.299788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36102 (local address 127.24.215.190:39031)
0810 14:49:36.300645 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:36.300662 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:36.300714 (+ 52us) server_negotiation.cc:407] Connection header received
0810 14:49:36.303838 (+ 3124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:36.303866 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:36.304365 (+ 499us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:36.304819 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:36.306279 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:36.308423 (+ 2144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:36.309885 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:36.310972 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:36.311215 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:36.315881 (+ 4666us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:36.315892 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:36.315911 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:36.315955 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:36.320015 (+ 4060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:36.320680 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:36.320689 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:36.320698 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:36.320914 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:36.321294 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:36.321299 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:36.321303 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:36.322077 (+ 774us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:36.322376 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:36.322791 (+ 415us) server_negotiation.cc:299] Negotiation successful
0810 14:49:36.323308 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":351,"threads_started":1}
I20260810 14:49:36.325644 15951 raft_consensus.cc:348] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f597ea5d9c5840d0be54308b286a76d7" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39031 } }
I20260810 14:49:36.326023 15951 raft_consensus.cc:374] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:36.326169 15951 raft_consensus.cc:729] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f597ea5d9c5840d0be54308b286a76d7, State: Initialized, Role: FOLLOWER
I20260810 14:49:36.326719 15951 consensus_queue.cc:260] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f597ea5d9c5840d0be54308b286a76d7" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39031 } }
I20260810 14:49:36.326984 15951 raft_consensus.cc:388] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:49:36.327167 15951 raft_consensus.cc:482] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:49:36.327374 15951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:36.329568 15951 raft_consensus.cc:504] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f597ea5d9c5840d0be54308b286a76d7" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39031 } }
I20260810 14:49:36.330073 15951 leader_election.cc:302] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f597ea5d9c5840d0be54308b286a76d7; no voters:
I20260810 14:49:36.330665 15951 leader_election.cc:288] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:49:36.330816 15957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:36.331660 15957 raft_consensus.cc:686] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [term 1 LEADER]: Becoming Leader. State: Replica: f597ea5d9c5840d0be54308b286a76d7, State: Running, Role: LEADER
I20260810 14:49:36.332276 15957 consensus_queue.cc:237] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f597ea5d9c5840d0be54308b286a76d7" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39031 } }
I20260810 14:49:36.332876 15951 sys_catalog.cc:564] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:49:36.335745 15958 sys_catalog.cc:455] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f597ea5d9c5840d0be54308b286a76d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f597ea5d9c5840d0be54308b286a76d7" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39031 } } }
I20260810 14:49:36.335772 15960 sys_catalog.cc:455] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f597ea5d9c5840d0be54308b286a76d7. Latest consensus state: current_term: 1 leader_uuid: "f597ea5d9c5840d0be54308b286a76d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f597ea5d9c5840d0be54308b286a76d7" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39031 } } }
I20260810 14:49:36.336237 15958 sys_catalog.cc:458] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:36.336237 15960 sys_catalog.cc:458] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:36.343837 15967 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:49:36.348932 15967 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:49:36.357537 15967 catalog_manager.cc:1347] Generated new cluster ID: 0ec9308219bd403f8d64da748495fd50
I20260810 14:49:36.357657 15967 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:49:36.369238 15967 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:49:36.371145 15967 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:49:36.389780 15967 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7: Generated new TSK 0
I20260810 14:49:36.390605 15967 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.24.215.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
Entry for principal kudu/127.24.215.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab.
I20260810 14:49:36.472863 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:39031
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:36.814787 15980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:36.815069 15980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:36.815253 15980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:36.822566 15980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:36.822679 15980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:36.822794 15980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:49:36.832789 15980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.129.keytab
--principal=kudu/127.24.215.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:39031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:36.834247 15980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:36.835747 15980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:36.849197 15986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:36.849282 15985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:36.849304 15988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:36.849997 15980 server_base.cc:1034] running on GCE node
I20260810 14:49:36.850632 15980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:36.851624 15980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:36.852842 15980 hybrid_clock.cc:648] HybridClock initialized: now 1786373376852774 us; error 52 us; skew 500 ppm
I20260810 14:49:36.853176 15980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:36 dist-test-slave-vkjp krb5kdc[15858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373376, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:36.856328 15980 init.cc:377] Logged in from keytab as kudu/127.24.215.129@KRBTEST.COM (short username kudu)
I20260810 14:49:36.857738 15980 webserver.cc:460] Webserver started at http://127.24.215.129:41711/ using document root <none> and password file <none>
I20260810 14:49:36.858433 15980 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:36.858584 15980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:36.858919 15980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:36.861904 15980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "753456ce78d04553a941ab4dc8c3fc5e"
format_stamp: "Formatted at 2026-08-10 14:49:36 on dist-test-slave-vkjp"
server_key: "d828ad1826ff84ad43133a6a4aee1165"
server_key_iv: "8cc4bc2861adf80810716f8d395e3381"
server_key_version: "encryptionkey@0"
I20260810 14:49:36.862704 15980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "753456ce78d04553a941ab4dc8c3fc5e"
format_stamp: "Formatted at 2026-08-10 14:49:36 on dist-test-slave-vkjp"
server_key: "d828ad1826ff84ad43133a6a4aee1165"
server_key_iv: "8cc4bc2861adf80810716f8d395e3381"
server_key_version: "encryptionkey@0"
I20260810 14:49:36.868284 15980 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260810 14:49:36.872020 15996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:36.873375 15980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 14:49:36.873564 15980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "753456ce78d04553a941ab4dc8c3fc5e"
format_stamp: "Formatted at 2026-08-10 14:49:36 on dist-test-slave-vkjp"
server_key: "d828ad1826ff84ad43133a6a4aee1165"
server_key_iv: "8cc4bc2861adf80810716f8d395e3381"
server_key_version: "encryptionkey@0"
I20260810 14:49:36.873793 15980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:36.903923 15980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:36.907969 15980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:36.908288 15980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:36.909296 15980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:36.910678 15980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:36.910789 15980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:36.910953 15980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:36.911041 15980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:36.944905 15980 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:37503
I20260810 14:49:36.944949 16130 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:37503 every 8 connection(s)
I20260810 14:49:36.946205 15980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:49:36.952912 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 15980
I20260810 14:49:36.953111 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:49:36.953574 25438 external_mini_cluster.cc:1428] Setting key f20287320cd5ae876939104060c43b4f
Aug 10 14:49:36 dist-test-slave-vkjp krb5kdc[15858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373376, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.129@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:49:36.967715 16134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:36.948972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:49265 (local address 127.24.215.190:39031)
0810 14:49:36.949480 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:36.949486 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:36.950318 (+ 832us) server_negotiation.cc:407] Connection header received
0810 14:49:36.952292 (+ 1974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:36.952298 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:36.952396 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:36.952518 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:36.954264 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:36.955053 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:36.956810 (+ 1757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:36.957238 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:36.957376 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:36.961554 (+ 4178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:36.961559 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:36.961563 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:36.961596 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:36.964201 (+ 2605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:36.965162 (+ 961us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:36.965165 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:36.965168 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:36.965283 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:36.965785 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:36.965788 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:36.965790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:36.966045 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:36.966153 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:36.967148 (+ 995us) server_negotiation.cc:299] Negotiation successful
0810 14:49:36.967398 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":277,"threads_started":1}
I20260810 14:49:36.968736 16135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:36.949446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:39031 (local address 127.24.215.129:49265)
0810 14:49:36.950156 (+ 710us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:36.950195 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:36.952008 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:36.952767 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:36.952776 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:36.953393 (+ 617us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:36.954032 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:36.954052 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:36.955243 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:36.955248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:36.956654 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:36.956668 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:36.957499 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:36.957506 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:36.957751 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:36.958465 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:49:36.958489 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:36.961294 (+ 2805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:36.964369 (+ 3075us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:36.964380 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:36.964400 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:36.965040 (+ 640us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:36.965392 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:36.965399 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:36.965405 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:36.965671 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:36.966172 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:36.966183 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:36.966659 (+ 476us) client_negotiation.cc:769] Sending connection context
0810 14:49:36.966981 (+ 322us) client_negotiation.cc:240] Negotiation successful
0810 14:49:36.967629 (+ 648us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":298,"threads_started":1}
I20260810 14:49:36.970362 16131 heartbeater.cc:347] Connected to a master server at 127.24.215.190:39031
I20260810 14:49:36.970695 16131 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:36.971586 16131 heartbeater.cc:511] Master 127.24.215.190:39031 requested a full tablet report, sending...
I20260810 14:49:36.973892 15910 ts_manager.cc:194] Registered new tserver with Master: 753456ce78d04553a941ab4dc8c3fc5e (127.24.215.129:37503)
I20260810 14:49:36.975478 15910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.129@KRBTEST.COM'} at 127.24.215.129:49265
WARNING: no policy specified for kudu/127.24.215.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
Entry for principal kudu/127.24.215.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab.
I20260810 14:49:37.022001 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:39031
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:37.369104 16139 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:37.369421 16139 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:37.369609 16139 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:37.376899 16139 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:37.377050 16139 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:37.377249 16139 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:49:37.386564 16139 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.130.keytab
--principal=kudu/127.24.215.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:39031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:37.387929 16139 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:37.389235 16139 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:37.402406 16147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:37.402554 16145 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:37.402515 16144 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:37.403378 16139 server_base.cc:1034] running on GCE node
I20260810 14:49:37.403913 16139 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:37.404829 16139 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:37.406042 16139 hybrid_clock.cc:648] HybridClock initialized: now 1786373377406003 us; error 25 us; skew 500 ppm
I20260810 14:49:37.406374 16139 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:37 dist-test-slave-vkjp krb5kdc[15858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373377, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:37.409502 16139 init.cc:377] Logged in from keytab as kudu/127.24.215.130@KRBTEST.COM (short username kudu)
I20260810 14:49:37.410982 16139 webserver.cc:460] Webserver started at http://127.24.215.130:43301/ using document root <none> and password file <none>
I20260810 14:49:37.411695 16139 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:37.411832 16139 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:37.412176 16139 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:37.415030 16139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "ee8c524f6cd34bd28bf161799666111d"
format_stamp: "Formatted at 2026-08-10 14:49:37 on dist-test-slave-vkjp"
server_key: "cc3d09bc56eaeb802eae92bb08d3479e"
server_key_iv: "8915f42d0ddf7f0daf1ae5b36af2a0f2"
server_key_version: "encryptionkey@0"
I20260810 14:49:37.415861 16139 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "ee8c524f6cd34bd28bf161799666111d"
format_stamp: "Formatted at 2026-08-10 14:49:37 on dist-test-slave-vkjp"
server_key: "cc3d09bc56eaeb802eae92bb08d3479e"
server_key_iv: "8915f42d0ddf7f0daf1ae5b36af2a0f2"
server_key_version: "encryptionkey@0"
I20260810 14:49:37.421044 16139 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:49:37.424469 16154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:37.425786 16139 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:49:37.425976 16139 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "ee8c524f6cd34bd28bf161799666111d"
format_stamp: "Formatted at 2026-08-10 14:49:37 on dist-test-slave-vkjp"
server_key: "cc3d09bc56eaeb802eae92bb08d3479e"
server_key_iv: "8915f42d0ddf7f0daf1ae5b36af2a0f2"
server_key_version: "encryptionkey@0"
I20260810 14:49:37.426170 16139 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:37.448184 16139 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:37.452069 16139 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:37.452363 16139 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:37.453322 16139 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:37.454660 16139 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:37.454738 16139 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:37.454912 16139 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:37.454988 16139 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:37.488488 16139 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:44649
I20260810 14:49:37.488580 16281 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:44649 every 8 connection(s)
I20260810 14:49:37.489802 16139 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:49:37.491803 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 16139
I20260810 14:49:37.491999 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:49:37.492595 25438 external_mini_cluster.cc:1428] Setting key e61723967cc0c1aa0484b89122f96db4
Aug 10 14:49:37 dist-test-slave-vkjp krb5kdc[15858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373377, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.130@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:49:37.513233 16285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:37.492460 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:55475 (local address 127.24.215.190:39031)
0810 14:49:37.492957 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:37.492963 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:37.494021 (+ 1058us) server_negotiation.cc:407] Connection header received
0810 14:49:37.496733 (+ 2712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:37.496737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:37.496801 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:37.496886 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:37.499011 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:37.499808 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:37.502283 (+ 2475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:37.502698 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:37.502788 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:37.507028 (+ 4240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:37.507033 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:37.507036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:37.507069 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:37.509799 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:37.510556 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:37.510564 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:37.510571 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:37.510736 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:37.511214 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:37.511222 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:37.511229 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:37.511623 (+ 394us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:37.511780 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:37.512484 (+ 704us) server_negotiation.cc:299] Negotiation successful
0810 14:49:37.512747 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":245,"threads_started":1}
I20260810 14:49:37.513808 16286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:37.492982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:39031 (local address 127.24.215.130:55475)
0810 14:49:37.493864 (+ 882us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:37.493902 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:37.496414 (+ 2512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:37.497185 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:37.497201 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:37.497995 (+ 794us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:37.498811 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:37.498846 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:37.499976 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:37.499983 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:37.502129 (+ 2146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:37.502148 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:37.502929 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:37.502939 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:37.503337 (+ 398us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:37.504089 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:49:37.504126 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:37.506783 (+ 2657us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:37.509948 (+ 3165us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:37.509955 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:37.509967 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:37.510420 (+ 453us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:37.510848 (+ 428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:37.510851 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:37.510854 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:37.511038 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:37.511748 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:37.511754 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:37.512065 (+ 311us) client_negotiation.cc:769] Sending connection context
0810 14:49:37.512288 (+ 223us) client_negotiation.cc:240] Negotiation successful
0810 14:49:37.512682 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":373,"threads_started":1}
I20260810 14:49:37.515327 16282 heartbeater.cc:347] Connected to a master server at 127.24.215.190:39031
I20260810 14:49:37.515656 16282 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:37.516355 16282 heartbeater.cc:511] Master 127.24.215.190:39031 requested a full tablet report, sending...
I20260810 14:49:37.517750 15910 ts_manager.cc:194] Registered new tserver with Master: ee8c524f6cd34bd28bf161799666111d (127.24.215.130:44649)
I20260810 14:49:37.518580 15910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.130@KRBTEST.COM'} at 127.24.215.130:55475
WARNING: no policy specified for kudu/127.24.215.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
Entry for principal kudu/127.24.215.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab.
I20260810 14:49:37.562376 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:39031
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:43703
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:37.905479 16290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:37.905874 16290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:37.906041 16290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:37.913409 16290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:37.913573 16290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:37.913765 16290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:49:37.923887 16290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:43703
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.131.keytab
--principal=kudu/127.24.215.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:39031
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:37.925311 16290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:37.926676 16290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:37.940825 16295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:37.940752 16296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:37.940722 16298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:37.941669 16290 server_base.cc:1034] running on GCE node
I20260810 14:49:37.942339 16290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:37.943727 16290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:37.944947 16290 hybrid_clock.cc:648] HybridClock initialized: now 1786373377944895 us; error 37 us; skew 500 ppm
I20260810 14:49:37.945305 16290 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:37 dist-test-slave-vkjp krb5kdc[15858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373377, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:37.948661 16290 init.cc:377] Logged in from keytab as kudu/127.24.215.131@KRBTEST.COM (short username kudu)
I20260810 14:49:37.950189 16290 webserver.cc:460] Webserver started at http://127.24.215.131:46539/ using document root <none> and password file <none>
I20260810 14:49:37.950886 16290 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:37.951030 16290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:37.951391 16290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:37.954243 16290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "a2c9adcad72945a1b5109bab9e7ebae6"
format_stamp: "Formatted at 2026-08-10 14:49:37 on dist-test-slave-vkjp"
server_key: "c399599e3b64016351d70b7bbebf46c8"
server_key_iv: "9b409fac1cb84c2fd3036da772fb8250"
server_key_version: "encryptionkey@0"
I20260810 14:49:37.955055 16290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "a2c9adcad72945a1b5109bab9e7ebae6"
format_stamp: "Formatted at 2026-08-10 14:49:37 on dist-test-slave-vkjp"
server_key: "c399599e3b64016351d70b7bbebf46c8"
server_key_iv: "9b409fac1cb84c2fd3036da772fb8250"
server_key_version: "encryptionkey@0"
I20260810 14:49:37.960390 16290 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 14:49:37.964007 16305 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:37.965355 16290 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260810 14:49:37.965531 16290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "a2c9adcad72945a1b5109bab9e7ebae6"
format_stamp: "Formatted at 2026-08-10 14:49:37 on dist-test-slave-vkjp"
server_key: "c399599e3b64016351d70b7bbebf46c8"
server_key_iv: "9b409fac1cb84c2fd3036da772fb8250"
server_key_version: "encryptionkey@0"
I20260810 14:49:37.965757 16290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:37.979251 16131 heartbeater.cc:503] Master 127.24.215.190:39031 was elected leader, sending a full tablet report...
I20260810 14:49:37.981930 16290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:37.986400 16290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:37.986719 16290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:37.987840 16290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:37.989255 16290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:37.989331 16290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:37.989504 16290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:37.989571 16290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:38.025681 16290 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:40861
I20260810 14:49:38.025727 16430 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:40861 every 8 connection(s)
I20260810 14:49:38.027027 16290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:49:38.032498 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 16290
I20260810 14:49:38.032704 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:49:38.033332 25438 external_mini_cluster.cc:1428] Setting key e9b373b4114e2b497bfd215194956ce2
Aug 10 14:49:38 dist-test-slave-vkjp krb5kdc[15858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373377, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.131@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:49:38.047857 16433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.029892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:58925 (local address 127.24.215.190:39031)
0810 14:49:38.030472 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:38.030478 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:38.031308 (+ 830us) server_negotiation.cc:407] Connection header received
0810 14:49:38.033435 (+ 2127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:38.033444 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:38.033573 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:38.033712 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:38.035570 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.036401 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.037938 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.038344 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.038437 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.041771 (+ 3334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:38.041776 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:38.041780 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:38.041812 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:38.044808 (+ 2996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:38.045535 (+ 727us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:38.045539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:38.045542 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:38.045668 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:38.046060 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:38.046063 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:38.046065 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:38.046312 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:38.046433 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:38.047201 (+ 768us) server_negotiation.cc:299] Negotiation successful
0810 14:49:38.047437 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":474,"thread_start_us":294,"threads_started":1}
I20260810 14:49:38.048934 16434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.030187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:39031 (local address 127.24.215.131:58925)
0810 14:49:38.031135 (+ 948us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:38.031188 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:38.033094 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:38.033976 (+ 882us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:38.033987 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:38.034593 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:38.035394 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:38.035416 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.036560 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.036564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:38.037806 (+ 1242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:38.037819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.038529 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.038534 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:38.038736 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.039403 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:49:38.039425 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:38.041582 (+ 2157us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:38.044938 (+ 3356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:38.044946 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:38.044958 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:38.045398 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:38.045763 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:49:38.045766 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:49:38.045768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:49:38.045963 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:49:38.046427 (+ 464us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:38.046432 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:38.046741 (+ 309us) client_negotiation.cc:769] Sending connection context
0810 14:49:38.047045 (+ 304us) client_negotiation.cc:240] Negotiation successful
0810 14:49:38.047637 (+ 592us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":409,"threads_started":1}
I20260810 14:49:38.050299 16431 heartbeater.cc:347] Connected to a master server at 127.24.215.190:39031
I20260810 14:49:38.050598 16431 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:38.051366 16431 heartbeater.cc:511] Master 127.24.215.190:39031 requested a full tablet report, sending...
I20260810 14:49:38.052711 15910 ts_manager.cc:194] Registered new tserver with Master: a2c9adcad72945a1b5109bab9e7ebae6 (127.24.215.131:40861)
I20260810 14:49:38.053575 15910 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.24.215.131@KRBTEST.COM'} at 127.24.215.131:58925
I20260810 14:49:38.061295 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:49:38.078390 16433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.065461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36104 (local address 127.24.215.190:39031)
0810 14:49:38.065641 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:38.065645 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:38.066138 (+ 493us) server_negotiation.cc:407] Connection header received
0810 14:49:38.066275 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:38.066279 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:38.066362 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:38.066447 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:38.067968 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.068700 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.070162 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.070639 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.070753 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.073078 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:38.073083 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:38.073086 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:38.073117 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:38.075830 (+ 2713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:38.076525 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:38.076529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:38.076531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:38.076658 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:38.077030 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:38.077033 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:38.077035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:38.077407 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:38.077597 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:38.077933 (+ 336us) server_negotiation.cc:299] Negotiation successful
0810 14:49:38.078102 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 14:49:38.082389 15910 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:49:38.089370 15910 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:49:38.114827 16451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.105186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:40861 (local address 127.0.0.1:40086)
0810 14:49:38.106892 (+ 1706us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:38.106909 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:38.107048 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:38.107707 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:38.107711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:38.107736 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:38.108073 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:38.108081 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.109743 (+ 1662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.109747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:38.111619 (+ 1872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:38.111632 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.113718 (+ 2086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.113725 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:38.113928 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.113984 (+ 56us) client_negotiation.cc:769] Sending connection context
0810 14:49:38.114113 (+ 129us) client_negotiation.cc:240] Negotiation successful
0810 14:49:38.114303 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1539,"mutex_wait_us":35,"spinlock_wait_cycles":6016,"thread_start_us":380,"threads_started":1}
I20260810 14:49:38.115446 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.102254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36238 (local address 127.24.215.129:37503)
0810 14:49:38.103101 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:38.103111 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:38.104762 (+ 1651us) server_negotiation.cc:407] Connection header received
0810 14:49:38.105857 (+ 1095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:38.105867 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:38.106153 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:38.106323 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:38.107449 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.109143 (+ 1694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.112428 (+ 3285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.113942 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.114149 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.114308 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:38.114903 (+ 595us) server_negotiation.cc:299] Negotiation successful
0810 14:49:38.115066 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"spinlock_wait_cycles":19712,"thread_start_us":551,"threads_started":1}
I20260810 14:49:38.115849 16452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.105470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40086 (local address 127.24.215.131:40861)
0810 14:49:38.106636 (+ 1166us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:38.106642 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:38.107199 (+ 557us) server_negotiation.cc:407] Connection header received
0810 14:49:38.107308 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:38.107316 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:38.107527 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:38.107724 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:38.108224 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.109566 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.111764 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.113197 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.113391 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.113579 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:38.115214 (+ 1635us) server_negotiation.cc:299] Negotiation successful
0810 14:49:38.115419 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1023,"thread_start_us":865,"threads_started":1}
I20260810 14:49:38.115417 16447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.102256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:37503 (local address 127.0.0.1:36238)
0810 14:49:38.104565 (+ 2309us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:38.104611 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:38.104881 (+ 270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:38.106659 (+ 1778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:38.106669 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:38.106705 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:38.107278 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:38.107295 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.109294 (+ 1999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.109299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:38.112252 (+ 2953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:38.112268 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.114567 (+ 2299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.114572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:38.114709 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.114732 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 14:49:38.114839 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 14:49:38.114978 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2070,"mutex_wait_us":195,"thread_start_us":1883,"threads_started":1}
I20260810 14:49:38.117060 16448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.102847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:44649 (local address 127.0.0.1:57298)
0810 14:49:38.105856 (+ 3009us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:38.105874 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:38.105995 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:38.106766 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:38.106771 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:38.106803 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:38.107383 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:38.107395 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.109294 (+ 1899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.109299 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:38.112253 (+ 2954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:38.112268 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.116230 (+ 3962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.116236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:38.116418 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.116450 (+ 32us) client_negotiation.cc:769] Sending connection context
0810 14:49:38.116568 (+ 118us) client_negotiation.cc:240] Negotiation successful
0810 14:49:38.116703 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2875,"mutex_wait_us":185,"thread_start_us":2259,"threads_started":1}
I20260810 14:49:38.117180 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.103486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57298 (local address 127.24.215.130:44649)
0810 14:49:38.104213 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:38.104226 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:38.106227 (+ 2001us) server_negotiation.cc:407] Connection header received
0810 14:49:38.106339 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:38.106345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:38.106563 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:38.106751 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:38.107570 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.109126 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.114402 (+ 5276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.115959 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.116140 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.116323 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:38.116655 (+ 332us) server_negotiation.cc:299] Negotiation successful
0810 14:49:38.116862 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":323,"threads_started":1}
I20260810 14:49:38.120210 16054 tablet_service.cc:1467] Processing CreateTablet for tablet 059c748072c3452ea3c6ccd9cd618a43 (DEFAULT_TABLE table=test-table [id=bd2ccfb00f5c4321a8f6db6f0db374cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:38.120502 16360 tablet_service.cc:1467] Processing CreateTablet for tablet 059c748072c3452ea3c6ccd9cd618a43 (DEFAULT_TABLE table=test-table [id=bd2ccfb00f5c4321a8f6db6f0db374cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:38.120508 16210 tablet_service.cc:1467] Processing CreateTablet for tablet 059c748072c3452ea3c6ccd9cd618a43 (DEFAULT_TABLE table=test-table [id=bd2ccfb00f5c4321a8f6db6f0db374cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:38.121551 16054 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 059c748072c3452ea3c6ccd9cd618a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:38.121862 16360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 059c748072c3452ea3c6ccd9cd618a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:38.121874 16210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 059c748072c3452ea3c6ccd9cd618a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:38.133615 16453 tablet_bootstrap.cc:492] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e: Bootstrap starting.
I20260810 14:49:38.137356 16453 tablet_bootstrap.cc:654] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:38.138473 16453 log.cc:824] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:38.142354 16454 tablet_bootstrap.cc:492] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d: Bootstrap starting.
I20260810 14:49:38.143268 16455 tablet_bootstrap.cc:492] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6: Bootstrap starting.
I20260810 14:49:38.146018 16454 tablet_bootstrap.cc:654] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:38.146739 16455 tablet_bootstrap.cc:654] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:38.147449 16454 log.cc:824] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:38.148730 16455 log.cc:824] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:38.149714 16453 tablet_bootstrap.cc:492] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e: No bootstrap required, opened a new log
I20260810 14:49:38.150007 16453 ts_tablet_manager.cc:1397] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e: Time spent bootstrapping tablet: real 0.017s user 0.009s sys 0.005s
I20260810 14:49:38.150381 16454 tablet_bootstrap.cc:492] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d: No bootstrap required, opened a new log
I20260810 14:49:38.150652 16454 ts_tablet_manager.cc:1397] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260810 14:49:38.151806 16455 tablet_bootstrap.cc:492] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6: No bootstrap required, opened a new log
I20260810 14:49:38.152104 16455 ts_tablet_manager.cc:1397] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260810 14:49:38.157928 16454 raft_consensus.cc:348] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753456ce78d04553a941ab4dc8c3fc5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 37503 } } peers { permanent_uuid: "a2c9adcad72945a1b5109bab9e7ebae6" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 40861 } } peers { permanent_uuid: "ee8c524f6cd34bd28bf161799666111d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44649 } }
I20260810 14:49:38.158306 16454 raft_consensus.cc:374] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:38.158429 16454 raft_consensus.cc:729] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee8c524f6cd34bd28bf161799666111d, State: Initialized, Role: FOLLOWER
I20260810 14:49:38.159067 16454 consensus_queue.cc:260] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753456ce78d04553a941ab4dc8c3fc5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 37503 } } peers { permanent_uuid: "a2c9adcad72945a1b5109bab9e7ebae6" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 40861 } } peers { permanent_uuid: "ee8c524f6cd34bd28bf161799666111d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44649 } }
I20260810 14:49:38.160308 16282 heartbeater.cc:503] Master 127.24.215.190:39031 was elected leader, sending a full tablet report...
I20260810 14:49:38.161111 16454 ts_tablet_manager.cc:1428] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d: Time spent starting tablet: real 0.010s user 0.005s sys 0.004s
I20260810 14:49:38.161283 16453 raft_consensus.cc:348] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753456ce78d04553a941ab4dc8c3fc5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 37503 } } peers { permanent_uuid: "a2c9adcad72945a1b5109bab9e7ebae6" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 40861 } } peers { permanent_uuid: "ee8c524f6cd34bd28bf161799666111d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44649 } }
I20260810 14:49:38.161715 16453 raft_consensus.cc:374] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:38.161823 16453 raft_consensus.cc:729] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 753456ce78d04553a941ab4dc8c3fc5e, State: Initialized, Role: FOLLOWER
I20260810 14:49:38.162628 16453 consensus_queue.cc:260] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753456ce78d04553a941ab4dc8c3fc5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 37503 } } peers { permanent_uuid: "a2c9adcad72945a1b5109bab9e7ebae6" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 40861 } } peers { permanent_uuid: "ee8c524f6cd34bd28bf161799666111d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44649 } }
I20260810 14:49:38.162851 16455 raft_consensus.cc:348] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753456ce78d04553a941ab4dc8c3fc5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 37503 } } peers { permanent_uuid: "a2c9adcad72945a1b5109bab9e7ebae6" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 40861 } } peers { permanent_uuid: "ee8c524f6cd34bd28bf161799666111d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44649 } }
I20260810 14:49:38.163431 16455 raft_consensus.cc:374] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:38.163587 16455 raft_consensus.cc:729] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2c9adcad72945a1b5109bab9e7ebae6, State: Initialized, Role: FOLLOWER
I20260810 14:49:38.164368 16455 consensus_queue.cc:260] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753456ce78d04553a941ab4dc8c3fc5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 37503 } } peers { permanent_uuid: "a2c9adcad72945a1b5109bab9e7ebae6" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 40861 } } peers { permanent_uuid: "ee8c524f6cd34bd28bf161799666111d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44649 } }
I20260810 14:49:38.164623 16453 ts_tablet_manager.cc:1428] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e: Time spent starting tablet: real 0.014s user 0.013s sys 0.000s
I20260810 14:49:38.166787 16431 heartbeater.cc:503] Master 127.24.215.190:39031 was elected leader, sending a full tablet report...
I20260810 14:49:38.167484 16455 ts_tablet_manager.cc:1428] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
W20260810 14:49:38.198912 16132 tablet.cc:2369] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:49:38.241868 16283 tablet.cc:2369] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:49:38.278802 16432 tablet.cc:2369] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:49:38.390333 16459 raft_consensus.cc:482] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:49:38.390753 16459 raft_consensus.cc:504] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753456ce78d04553a941ab4dc8c3fc5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 37503 } } peers { permanent_uuid: "a2c9adcad72945a1b5109bab9e7ebae6" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 40861 } } peers { permanent_uuid: "ee8c524f6cd34bd28bf161799666111d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44649 } }
I20260810 14:49:38.392563 16459 leader_election.cc:288] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 753456ce78d04553a941ab4dc8c3fc5e (127.24.215.129:37503), a2c9adcad72945a1b5109bab9e7ebae6 (127.24.215.131:40861)
I20260810 14:49:38.399879 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.393280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:53157 (local address 127.24.215.129:37503)
0810 14:49:38.393536 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:38.393543 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:38.393776 (+ 233us) server_negotiation.cc:407] Connection header received
0810 14:49:38.394011 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:38.394020 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:38.394129 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:38.394231 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:38.395113 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.396098 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.398006 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.398837 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.399035 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.399139 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:38.399490 (+ 351us) server_negotiation.cc:299] Negotiation successful
0810 14:49:38.399624 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260810 14:49:38.400065 16464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.393029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:37503 (local address 127.24.215.130:53157)
0810 14:49:38.393650 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:38.393666 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:38.393854 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:38.394414 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:38.394419 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:38.394459 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:38.394918 (+ 459us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:38.394932 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.396228 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.396235 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:38.397839 (+ 1604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:38.397852 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.399066 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.399072 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:38.399316 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.399356 (+ 40us) client_negotiation.cc:769] Sending connection context
0810 14:49:38.399467 (+ 111us) client_negotiation.cc:240] Negotiation successful
0810 14:49:38.399612 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"mutex_wait_us":36,"thread_start_us":327,"threads_started":1}
I20260810 14:49:38.400179 16452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.394137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:35181 (local address 127.24.215.131:40861)
0810 14:49:38.394349 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:38.394355 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:38.394569 (+ 214us) server_negotiation.cc:407] Connection header received
0810 14:49:38.394771 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:38.394776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:38.394853 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:38.394960 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:38.395539 (+ 579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.396418 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.398191 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.399195 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.399357 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.399436 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:38.399751 (+ 315us) server_negotiation.cc:299] Negotiation successful
0810 14:49:38.399875 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260810 14:49:38.400534 16465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.393798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:40861 (local address 127.24.215.130:35181)
0810 14:49:38.394450 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:38.394468 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:38.394616 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:38.394985 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:38.394990 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:38.395015 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:38.395364 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:38.395373 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.396566 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.396571 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:38.398012 (+ 1441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:38.398024 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.399418 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.399424 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:38.399591 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.399615 (+ 24us) client_negotiation.cc:769] Sending connection context
0810 14:49:38.399714 (+ 99us) client_negotiation.cc:240] Negotiation successful
0810 14:49:38.399842 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":6,"thread_start_us":308,"threads_started":1}
I20260810 14:49:38.401289 16078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "059c748072c3452ea3c6ccd9cd618a43" candidate_uuid: "ee8c524f6cd34bd28bf161799666111d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753456ce78d04553a941ab4dc8c3fc5e" is_pre_election: true
I20260810 14:49:38.401543 16383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "059c748072c3452ea3c6ccd9cd618a43" candidate_uuid: "ee8c524f6cd34bd28bf161799666111d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2c9adcad72945a1b5109bab9e7ebae6" is_pre_election: true
I20260810 14:49:38.401955 16078 raft_consensus.cc:2454] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee8c524f6cd34bd28bf161799666111d in term 0.
I20260810 14:49:38.402043 16383 raft_consensus.cc:2454] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ee8c524f6cd34bd28bf161799666111d in term 0.
I20260810 14:49:38.402734 16159 leader_election.cc:302] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2c9adcad72945a1b5109bab9e7ebae6, ee8c524f6cd34bd28bf161799666111d; no voters:
I20260810 14:49:38.403128 16459 raft_consensus.cc:2781] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:49:38.403312 16459 raft_consensus.cc:482] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:49:38.403463 16459 raft_consensus.cc:3037] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:38.405728 16459 raft_consensus.cc:504] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753456ce78d04553a941ab4dc8c3fc5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 37503 } } peers { permanent_uuid: "a2c9adcad72945a1b5109bab9e7ebae6" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 40861 } } peers { permanent_uuid: "ee8c524f6cd34bd28bf161799666111d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44649 } }
I20260810 14:49:38.406345 16459 leader_election.cc:288] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [CANDIDATE]: Term 1 election: Requested vote from peers 753456ce78d04553a941ab4dc8c3fc5e (127.24.215.129:37503), a2c9adcad72945a1b5109bab9e7ebae6 (127.24.215.131:40861)
I20260810 14:49:38.406899 16078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "059c748072c3452ea3c6ccd9cd618a43" candidate_uuid: "ee8c524f6cd34bd28bf161799666111d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "753456ce78d04553a941ab4dc8c3fc5e"
I20260810 14:49:38.406977 16383 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "059c748072c3452ea3c6ccd9cd618a43" candidate_uuid: "ee8c524f6cd34bd28bf161799666111d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a2c9adcad72945a1b5109bab9e7ebae6"
I20260810 14:49:38.407135 16078 raft_consensus.cc:3037] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:38.407277 16383 raft_consensus.cc:3037] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:38.409471 16078 raft_consensus.cc:2454] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee8c524f6cd34bd28bf161799666111d in term 1.
I20260810 14:49:38.409499 16383 raft_consensus.cc:2454] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ee8c524f6cd34bd28bf161799666111d in term 1.
I20260810 14:49:38.409917 16159 leader_election.cc:302] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a2c9adcad72945a1b5109bab9e7ebae6, ee8c524f6cd34bd28bf161799666111d; no voters:
I20260810 14:49:38.410235 16459 raft_consensus.cc:2781] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:38.410718 16459 raft_consensus.cc:686] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [term 1 LEADER]: Becoming Leader. State: Replica: ee8c524f6cd34bd28bf161799666111d, State: Running, Role: LEADER
I20260810 14:49:38.411284 16459 consensus_queue.cc:237] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753456ce78d04553a941ab4dc8c3fc5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 37503 } } peers { permanent_uuid: "a2c9adcad72945a1b5109bab9e7ebae6" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 40861 } } peers { permanent_uuid: "ee8c524f6cd34bd28bf161799666111d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44649 } }
I20260810 14:49:38.416383 15910 catalog_manager.cc:5495] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d reported cstate change: term changed from 0 to 1, leader changed from <none> to ee8c524f6cd34bd28bf161799666111d (127.24.215.130). New cstate: current_term: 1 leader_uuid: "ee8c524f6cd34bd28bf161799666111d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "753456ce78d04553a941ab4dc8c3fc5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 37503 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a2c9adcad72945a1b5109bab9e7ebae6" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 40861 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ee8c524f6cd34bd28bf161799666111d" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 44649 } health_report { overall_health: HEALTHY } } }
I20260810 14:49:38.478902 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.473252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57300 (local address 127.24.215.130:44649)
0810 14:49:38.473485 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:38.473493 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:38.473512 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:49:38.473749 (+ 237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:38.473753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:38.473844 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:38.474003 (+ 159us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:49:38.474428 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.475228 (+ 800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.476617 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.477079 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.477226 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.477754 (+ 528us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:49:38.478271 (+ 517us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:49:38.478450 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:38.478532 (+ 82us) server_negotiation.cc:299] Negotiation successful
0810 14:49:38.478619 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:49:38.486958 16383 raft_consensus.cc:1261] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6 [term 1 FOLLOWER]: Refusing update from remote peer ee8c524f6cd34bd28bf161799666111d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:49:38.487042 16078 raft_consensus.cc:1261] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e [term 1 FOLLOWER]: Refusing update from remote peer ee8c524f6cd34bd28bf161799666111d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:49:38.488041 16466 consensus_queue.cc:1035] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [LEADER]: Connected to new peer: Peer: permanent_uuid: "a2c9adcad72945a1b5109bab9e7ebae6" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 40861 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:49:38.488457 16459 consensus_queue.cc:1035] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [LEADER]: Connected to new peer: Peer: permanent_uuid: "753456ce78d04553a941ab4dc8c3fc5e" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 37503 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:49:38.506072 16472 mvcc.cc:205] Tried to move back new op lower bound from 7316985358273048576 to 7316985357976743936. Current Snapshot: MvccSnapshot[applied={T|T < 7316985358273048576}]
I20260810 14:49:38.509171 16471 mvcc.cc:205] Tried to move back new op lower bound from 7316985358273048576 to 7316985357976743936. Current Snapshot: MvccSnapshot[applied={T|T < 7316985358273048576}]
I20260810 14:49:38.520462 15910 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:49:38.520743 15910 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:49:38.524219 15910 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f597ea5d9c5840d0be54308b286a76d7: Sending DeleteTablet for 3 replicas of tablet 059c748072c3452ea3c6ccd9cd618a43
I20260810 14:49:38.525385 16054 tablet_service.cc:1514] Processing DeleteTablet for tablet 059c748072c3452ea3c6ccd9cd618a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:38 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:36238
I20260810 14:49:38.525578 16360 tablet_service.cc:1514] Processing DeleteTablet for tablet 059c748072c3452ea3c6ccd9cd618a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:38 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:40086
I20260810 14:49:38.525658 16210 tablet_service.cc:1514] Processing DeleteTablet for tablet 059c748072c3452ea3c6ccd9cd618a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:38 UTC) from {username='kudu', principal='kudu/127.24.215.190@KRBTEST.COM'} at 127.0.0.1:57298
I20260810 14:49:38.526850 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 15980
I20260810 14:49:38.536826 16483 tablet_replica.cc:330] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d: stopping tablet replica
I20260810 14:49:38.537405 16483 raft_consensus.cc:2229] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:49:38.538182 16483 raft_consensus.cc:2258] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:38.539157 16482 tablet_replica.cc:330] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6: stopping tablet replica
I20260810 14:49:38.539597 16481 tablet_replica.cc:330] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e: stopping tablet replica
I20260810 14:49:38.539647 16482 raft_consensus.cc:2229] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:49:38.540005 16481 raft_consensus.cc:2229] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:49:38.540305 16482 raft_consensus.cc:2258] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:38.540508 16481 raft_consensus.cc:2258] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:38.540637 16483 ts_tablet_manager.cc:1905] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:38.542680 16481 ts_tablet_manager.cc:1905] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:38.544090 16482 ts_tablet_manager.cc:1905] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:38.550830 16481 ts_tablet_manager.cc:1918] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:49:38.551144 16481 log.cc:1196] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-0/wal/wals/059c748072c3452ea3c6ccd9cd618a43
I20260810 14:49:38.551908 16481 ts_tablet_manager.cc:1939] T 059c748072c3452ea3c6ccd9cd618a43 P 753456ce78d04553a941ab4dc8c3fc5e: Deleting consensus metadata
Aug 10 14:49:38 dist-test-slave-vkjp krb5kdc[15858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.129@KRBTEST.COM
I20260810 14:49:38.554164 15893 catalog_manager.cc:4841] TS 753456ce78d04553a941ab4dc8c3fc5e (127.24.215.129:37503): tablet 059c748072c3452ea3c6ccd9cd618a43 (table test-table [id=bd2ccfb00f5c4321a8f6db6f0db374cb]) successfully deleted
I20260810 14:49:38.556605 16483 ts_tablet_manager.cc:1918] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:49:38.556859 16483 log.cc:1196] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/059c748072c3452ea3c6ccd9cd618a43
I20260810 14:49:38.557564 16483 ts_tablet_manager.cc:1939] T 059c748072c3452ea3c6ccd9cd618a43 P ee8c524f6cd34bd28bf161799666111d: Deleting consensus metadata
I20260810 14:49:38.557773 16482 ts_tablet_manager.cc:1918] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:49:38.558008 16482 log.cc:1196] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/059c748072c3452ea3c6ccd9cd618a43
I20260810 14:49:38.558668 16482 ts_tablet_manager.cc:1939] T 059c748072c3452ea3c6ccd9cd618a43 P a2c9adcad72945a1b5109bab9e7ebae6: Deleting consensus metadata
I20260810 14:49:38.559703 15892 catalog_manager.cc:4841] TS ee8c524f6cd34bd28bf161799666111d (127.24.215.130:44649): tablet 059c748072c3452ea3c6ccd9cd618a43 (table test-table [id=bd2ccfb00f5c4321a8f6db6f0db374cb]) successfully deleted
I20260810 14:49:38.560508 15893 catalog_manager.cc:4841] TS a2c9adcad72945a1b5109bab9e7ebae6 (127.24.215.131:40861): tablet 059c748072c3452ea3c6ccd9cd618a43 (table test-table [id=bd2ccfb00f5c4321a8f6db6f0db374cb]) successfully deleted
I20260810 14:49:38.561573 16446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.528588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36250 (local address 127.24.215.129:37503)
0810 14:49:38.528820 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:38.528829 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:38.541289 (+ 12460us) server_negotiation.cc:407] Connection header received
0810 14:49:38.541615 (+ 326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:38.541621 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:38.541709 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:38.541789 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:38.543817 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.544987 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.546948 (+ 1961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.547641 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.547753 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.553581 (+ 5828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:38.553594 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:38.553612 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:38.553659 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:38.557684 (+ 4025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:38.558656 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:38.558666 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:38.558669 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:38.558848 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:38.559399 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:38.559407 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:38.559414 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:38.559911 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:38.560247 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:38.560940 (+ 693us) server_negotiation.cc:299] Negotiation successful
0810 14:49:38.561192 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260810 14:49:38.562247 16126 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:38.684012 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 15980
I20260810 14:49:38.706933 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 16139
Aug 10 14:49:38 dist-test-slave-vkjp krb5kdc[15858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.130@KRBTEST.COM
I20260810 14:49:38.723577 16450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.707810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57304 (local address 127.24.215.130:44649)
0810 14:49:38.708273 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:38.708278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:38.708294 (+ 16us) server_negotiation.cc:407] Connection header received
0810 14:49:38.708352 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:38.708356 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:38.708452 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:38.708576 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:38.709938 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.710748 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.712272 (+ 1524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.712737 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.712880 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.717228 (+ 4348us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:38.717247 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:38.717264 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:38.717305 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:38.720515 (+ 3210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:38.721274 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:38.721281 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:38.721284 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:38.721444 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:38.721852 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:38.721857 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:38.721860 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:38.722229 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:38.722591 (+ 362us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:38.722931 (+ 340us) server_negotiation.cc:299] Negotiation successful
0810 14:49:38.723204 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348}
I20260810 14:49:38.724274 16277 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:38.847632 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 16139
I20260810 14:49:38.871225 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 16290
Aug 10 14:49:38 dist-test-slave-vkjp krb5kdc[15858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373375, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.131@KRBTEST.COM
I20260810 14:49:38.890571 16452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:38.872077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40094 (local address 127.24.215.131:40861)
0810 14:49:38.872256 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:38.872261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:38.872332 (+ 71us) server_negotiation.cc:407] Connection header received
0810 14:49:38.872489 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:38.872493 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:38.872572 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:38.872664 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:38.873968 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.874705 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.876094 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:38.876486 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:38.876583 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:38.885386 (+ 8803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:38.885403 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:38.885420 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:38.885459 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:38.888126 (+ 2667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:38.888781 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:38.888789 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:38.888791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:38.888941 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:38.889259 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:38.889262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:38.889264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:38.889640 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:38.889863 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:38.890147 (+ 284us) server_negotiation.cc:299] Negotiation successful
0810 14:49:38.890317 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 14:49:38.891302 16426 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:38.982477 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 16290
I20260810 14:49:39.005086 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 15871
I20260810 14:49:39.006196 15945 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:39.118690 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 15871
2026-08-10T14:49:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4816 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-10T14:49:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:49:39Z Disabled control of system clock
I20260810 14:49:39.211186 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:34645
--webserver_interface=127.24.215.190
--webserver_port=0
--builtin_ntp_servers=127.24.215.148:38849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:34645
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:49:39.561959 16493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:39.562260 16493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:39.562423 16493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:39.569756 16493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:49:39.569885 16493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:39.569967 16493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:39.570082 16493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:49:39.570147 16493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:49:39.579411 16493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:38849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:34645
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:34645
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:39.580726 16493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:39.582549 16493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:39.595288 16502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:39.595566 16499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:39.596022 16500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:39.596947 16493 server_base.cc:1034] running on GCE node
I20260810 14:49:39.598006 16493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:39.599421 16493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:39.600657 16493 hybrid_clock.cc:648] HybridClock initialized: now 1786373379600601 us; error 52 us; skew 500 ppm
I20260810 14:49:39.601011 16493 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:49:39.603425 16493 webserver.cc:460] Webserver started at http://127.24.215.190:34479/ using document root <none> and password file <none>
I20260810 14:49:39.604110 16493 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:39.604242 16493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:39.604566 16493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:39.607538 16493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "de601544fbdc41c8b97f61008153ad4f"
format_stamp: "Formatted at 2026-08-10 14:49:39 on dist-test-slave-vkjp"
server_key: "2796b2776be14ae65cb18ead698754f5"
server_key_iv: "93904b381dba266e792b2e558a9336b1"
server_key_version: "encryptionkey@0"
I20260810 14:49:39.608356 16493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "de601544fbdc41c8b97f61008153ad4f"
format_stamp: "Formatted at 2026-08-10 14:49:39 on dist-test-slave-vkjp"
server_key: "2796b2776be14ae65cb18ead698754f5"
server_key_iv: "93904b381dba266e792b2e558a9336b1"
server_key_version: "encryptionkey@0"
I20260810 14:49:39.613404 16493 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:49:39.616786 16511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:39.618059 16493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 14:49:39.618256 16493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "de601544fbdc41c8b97f61008153ad4f"
format_stamp: "Formatted at 2026-08-10 14:49:39 on dist-test-slave-vkjp"
server_key: "2796b2776be14ae65cb18ead698754f5"
server_key_iv: "93904b381dba266e792b2e558a9336b1"
server_key_version: "encryptionkey@0"
I20260810 14:49:39.618474 16493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:39.637086 16493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:39.638051 16493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:39.638355 16493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:39.657874 16493 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:34645
I20260810 14:49:39.657895 16568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:34645 every 8 connection(s)
I20260810 14:49:39.659279 16493 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:49:39.663071 16569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:39.663784 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 16493
I20260810 14:49:39.663987 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:49:39.664443 25438 external_mini_cluster.cc:1428] Setting key 0dbc985d41cb60cc769ba48743ad7edf
I20260810 14:49:39.678249 16569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f: Bootstrap starting.
I20260810 14:49:39.680492 16572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:39.666354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58650 (local address 127.24.215.190:34645)
0810 14:49:39.667540 (+ 1186us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:39.667566 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:39.667632 (+ 66us) server_negotiation.cc:407] Connection header received
0810 14:49:39.670087 (+ 2455us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:39.670116 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:39.670770 (+ 654us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:39.671268 (+ 498us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:39.671824 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:39.673731 (+ 1907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:39.675149 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:39.676274 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:39.676456 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:39.677605 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:39.677621 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:39.677639 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:39.678363 (+ 724us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:39.678477 (+ 114us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:39.678502 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:39.678638 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:39.678835 (+ 197us) server_negotiation.cc:299] Negotiation successful
0810 14:49:39.679199 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":610,"thread_start_us":345,"threads_started":1}
I20260810 14:49:39.682081 16569 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:39.683244 16569 log.cc:824] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:39.686112 16569 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f: No bootstrap required, opened a new log
I20260810 14:49:39.693789 16569 raft_consensus.cc:348] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de601544fbdc41c8b97f61008153ad4f" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34645 } }
I20260810 14:49:39.694142 16569 raft_consensus.cc:374] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:39.694221 16569 raft_consensus.cc:729] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de601544fbdc41c8b97f61008153ad4f, State: Initialized, Role: FOLLOWER
I20260810 14:49:39.694708 16569 consensus_queue.cc:260] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de601544fbdc41c8b97f61008153ad4f" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34645 } }
I20260810 14:49:39.694958 16569 raft_consensus.cc:388] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:49:39.695178 16569 raft_consensus.cc:482] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:49:39.695386 16569 raft_consensus.cc:3037] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:39.697593 16569 raft_consensus.cc:504] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de601544fbdc41c8b97f61008153ad4f" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34645 } }
I20260810 14:49:39.698118 16569 leader_election.cc:302] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: de601544fbdc41c8b97f61008153ad4f; no voters:
I20260810 14:49:39.698729 16569 leader_election.cc:288] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:49:39.698869 16574 raft_consensus.cc:2781] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:39.699702 16574 raft_consensus.cc:686] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [term 1 LEADER]: Becoming Leader. State: Replica: de601544fbdc41c8b97f61008153ad4f, State: Running, Role: LEADER
I20260810 14:49:39.700313 16574 consensus_queue.cc:237] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de601544fbdc41c8b97f61008153ad4f" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34645 } }
I20260810 14:49:39.701131 16569 sys_catalog.cc:564] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:49:39.703141 16576 sys_catalog.cc:455] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [sys.catalog]: SysCatalogTable state changed. Reason: New leader de601544fbdc41c8b97f61008153ad4f. Latest consensus state: current_term: 1 leader_uuid: "de601544fbdc41c8b97f61008153ad4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de601544fbdc41c8b97f61008153ad4f" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34645 } } }
I20260810 14:49:39.703171 16575 sys_catalog.cc:455] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "de601544fbdc41c8b97f61008153ad4f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "de601544fbdc41c8b97f61008153ad4f" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34645 } } }
I20260810 14:49:39.703676 16575 sys_catalog.cc:458] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:39.704584 16576 sys_catalog.cc:458] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:39.708019 16579 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:49:39.712309 16579 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:49:39.721163 16579 catalog_manager.cc:1347] Generated new cluster ID: 58af5df3d8d74f9eb60966c17c40f9de
I20260810 14:49:39.721282 16579 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:49:39.733002 16579 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:49:39.734264 16579 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:49:39.739869 16579 catalog_manager.cc:5868] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f: Generated new TSK 0
I20260810 14:49:39.740618 16579 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:49:39.748255 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.129:0
--local_ip_for_outbound_sockets=127.24.215.129
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34645
--builtin_ntp_servers=127.24.215.148:38849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260810 14:49:40.091368 16594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:40.091696 16594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:40.091861 16594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:40.098922 16594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:40.099063 16594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:40.099252 16594 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.129
I20260810 14:49:40.108489 16594 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:38849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.24.215.129
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.129
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:40.109824 16594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:40.111637 16594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:40.124723 16602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:40.124693 16605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:40.124837 16603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:40.125422 16594 server_base.cc:1034] running on GCE node
I20260810 14:49:40.126065 16594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:40.127041 16594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:40.128299 16594 hybrid_clock.cc:648] HybridClock initialized: now 1786373380128262 us; error 28 us; skew 500 ppm
I20260810 14:49:40.128618 16594 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:49:40.130788 16594 webserver.cc:460] Webserver started at http://127.24.215.129:42603/ using document root <none> and password file <none>
I20260810 14:49:40.131534 16594 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:40.131654 16594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:40.131982 16594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:40.134825 16594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/data/instance:
uuid: "e7eb528413cb40acb7123a5d3ab372c5"
format_stamp: "Formatted at 2026-08-10 14:49:40 on dist-test-slave-vkjp"
server_key: "98a259e2040d72484e4d3677b60d9d23"
server_key_iv: "88d7bdaa402b3ff26dfb6b7b03d5c8ba"
server_key_version: "encryptionkey@0"
I20260810 14:49:40.135704 16594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance:
uuid: "e7eb528413cb40acb7123a5d3ab372c5"
format_stamp: "Formatted at 2026-08-10 14:49:40 on dist-test-slave-vkjp"
server_key: "98a259e2040d72484e4d3677b60d9d23"
server_key_iv: "88d7bdaa402b3ff26dfb6b7b03d5c8ba"
server_key_version: "encryptionkey@0"
I20260810 14:49:40.140908 16594 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:49:40.144538 16611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:40.145849 16594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260810 14:49:40.146045 16594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/wal
uuid: "e7eb528413cb40acb7123a5d3ab372c5"
format_stamp: "Formatted at 2026-08-10 14:49:40 on dist-test-slave-vkjp"
server_key: "98a259e2040d72484e4d3677b60d9d23"
server_key_iv: "88d7bdaa402b3ff26dfb6b7b03d5c8ba"
server_key_version: "encryptionkey@0"
I20260810 14:49:40.146278 16594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:40.185595 16594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:40.186475 16594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:40.186793 16594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:40.187909 16594 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:40.189453 16594 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:40.189587 16594 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:40.189714 16594 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:40.189803 16594 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:40.230405 16594 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.129:42239
I20260810 14:49:40.230458 16737 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.129:42239 every 8 connection(s)
I20260810 14:49:40.231819 16594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/data/info.pb
I20260810 14:49:40.238926 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 16594
I20260810 14:49:40.239180 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-0/wal/instance
I20260810 14:49:40.239739 25438 external_mini_cluster.cc:1428] Setting key b28873c82e27586264671c5d9c27b709
I20260810 14:49:40.242734 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.130:0
--local_ip_for_outbound_sockets=127.24.215.130
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34645
--builtin_ntp_servers=127.24.215.148:38849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:49:40.245455 16740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:40.234416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.129:42149 (local address 127.24.215.190:34645)
0810 14:49:40.234897 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:40.234904 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:40.236146 (+ 1242us) server_negotiation.cc:407] Connection header received
0810 14:49:40.237887 (+ 1741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:40.237892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:40.237958 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:40.238061 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:40.239117 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:40.240257 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:40.242379 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:40.242808 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:40.242960 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:40.244678 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:40.244684 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:40.244687 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:40.244705 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:40.244745 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:40.244757 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:40.244828 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:40.245090 (+ 262us) server_negotiation.cc:299] Negotiation successful
0810 14:49:40.245190 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":355,"thread_start_us":229,"threads_started":1}
I20260810 14:49:40.246935 16741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:40.235041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34645 (local address 127.24.215.129:42149)
0810 14:49:40.235988 (+ 947us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:40.236024 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:40.237644 (+ 1620us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:40.238259 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:40.238267 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:40.238362 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:40.238918 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:40.238934 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:40.240451 (+ 1517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:40.240456 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:40.242217 (+ 1761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:40.242230 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:40.243149 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:40.243159 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:40.243509 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:40.244296 (+ 787us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:49:40.244301 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:40.244357 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:49:40.244470 (+ 113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:40.244856 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:40.244869 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:40.244879 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:49:40.245064 (+ 185us) client_negotiation.cc:240] Negotiation successful
0810 14:49:40.245516 (+ 452us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":520,"thread_start_us":290,"threads_started":1}
I20260810 14:49:40.248600 16738 heartbeater.cc:347] Connected to a master server at 127.24.215.190:34645
I20260810 14:49:40.248947 16738 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:40.249732 16738 heartbeater.cc:511] Master 127.24.215.190:34645 requested a full tablet report, sending...
I20260810 14:49:40.251976 16529 ts_manager.cc:194] Registered new tserver with Master: e7eb528413cb40acb7123a5d3ab372c5 (127.24.215.129:42239)
I20260810 14:49:40.253446 16529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.215.129:42149
W20260810 14:49:40.614531 16742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:40.614953 16742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:40.615126 16742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:40.622611 16742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:40.622741 16742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:40.622912 16742 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.130
I20260810 14:49:40.632252 16742 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:38849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.24.215.130
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.130
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:40.633574 16742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:40.635464 16742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:40.648775 16750 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:40.648923 16753 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:40.648667 16751 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:40.650825 16742 server_base.cc:1034] running on GCE node
I20260810 14:49:40.651523 16742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:40.652571 16742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:40.653774 16742 hybrid_clock.cc:648] HybridClock initialized: now 1786373380653730 us; error 36 us; skew 500 ppm
I20260810 14:49:40.654183 16742 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:49:40.656479 16742 webserver.cc:460] Webserver started at http://127.24.215.130:40433/ using document root <none> and password file <none>
I20260810 14:49:40.657197 16742 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:40.657346 16742 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:40.657693 16742 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:40.660632 16742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/data/instance:
uuid: "f7576b90769842ccb5ef1e1189515c10"
format_stamp: "Formatted at 2026-08-10 14:49:40 on dist-test-slave-vkjp"
server_key: "fb1068fc19a605dab461b0d32783ce8b"
server_key_iv: "316e5e80c4f6d69d2cd544b386ed3620"
server_key_version: "encryptionkey@0"
I20260810 14:49:40.661438 16742 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance:
uuid: "f7576b90769842ccb5ef1e1189515c10"
format_stamp: "Formatted at 2026-08-10 14:49:40 on dist-test-slave-vkjp"
server_key: "fb1068fc19a605dab461b0d32783ce8b"
server_key_iv: "316e5e80c4f6d69d2cd544b386ed3620"
server_key_version: "encryptionkey@0"
I20260810 14:49:40.666896 16742 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 14:49:40.670495 16759 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:40.671914 16742 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:49:40.672102 16742 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/wal
uuid: "f7576b90769842ccb5ef1e1189515c10"
format_stamp: "Formatted at 2026-08-10 14:49:40 on dist-test-slave-vkjp"
server_key: "fb1068fc19a605dab461b0d32783ce8b"
server_key_iv: "316e5e80c4f6d69d2cd544b386ed3620"
server_key_version: "encryptionkey@0"
I20260810 14:49:40.672349 16742 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:40.687973 16742 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:40.688768 16742 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:40.689076 16742 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:40.690120 16742 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:40.691695 16742 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:40.691816 16742 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:40.691944 16742 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:40.692039 16742 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:40.725790 16742 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.130:37825
I20260810 14:49:40.725844 16884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.130:37825 every 8 connection(s)
I20260810 14:49:40.727157 16742 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/data/info.pb
I20260810 14:49:40.735852 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 16742
I20260810 14:49:40.736066 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/wal/instance
I20260810 14:49:40.736622 25438 external_mini_cluster.cc:1428] Setting key d13a42d6338c2ff09e4b9af90da9e4a1
I20260810 14:49:40.739872 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.24.215.131:0
--local_ip_for_outbound_sockets=127.24.215.131
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34645
--builtin_ntp_servers=127.24.215.148:38849
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260810 14:49:40.740561 16740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:40.729693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:38043 (local address 127.24.215.190:34645)
0810 14:49:40.729876 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:40.729881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:40.731258 (+ 1377us) server_negotiation.cc:407] Connection header received
0810 14:49:40.733044 (+ 1786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:40.733050 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:40.733159 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:40.733275 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:40.734402 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:40.735192 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:40.737542 (+ 2350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:40.738000 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:40.738099 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:40.739574 (+ 1475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:40.739579 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:40.739583 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:40.739600 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:40.739631 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:40.739643 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:40.739715 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:40.740152 (+ 437us) server_negotiation.cc:299] Negotiation successful
0810 14:49:40.740288 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:49:40.741874 16887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:40.730173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34645 (local address 127.24.215.130:38043)
0810 14:49:40.731043 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:40.731142 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:40.732776 (+ 1634us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:40.733535 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:40.733548 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:40.733661 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:40.734225 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:40.734241 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:40.735335 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:40.735342 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:40.737349 (+ 2007us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:40.737362 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:40.738226 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:40.738234 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:40.738520 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:40.739240 (+ 720us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:49:40.739244 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:40.739293 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:49:40.739386 (+ 93us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:40.739774 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:40.739785 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:40.739795 (+ 10us) client_negotiation.cc:769] Sending connection context
0810 14:49:40.740013 (+ 218us) client_negotiation.cc:240] Negotiation successful
0810 14:49:40.740478 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":473,"thread_start_us":278,"threads_started":1}
I20260810 14:49:40.743327 16885 heartbeater.cc:347] Connected to a master server at 127.24.215.190:34645
I20260810 14:49:40.743630 16885 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:40.744374 16885 heartbeater.cc:511] Master 127.24.215.190:34645 requested a full tablet report, sending...
I20260810 14:49:40.745792 16529 ts_manager.cc:194] Registered new tserver with Master: f7576b90769842ccb5ef1e1189515c10 (127.24.215.130:37825)
I20260810 14:49:40.746593 16529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.215.130:38043
W20260810 14:49:41.085314 16888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:41.085599 16888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:41.085757 16888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:41.092976 16888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:41.093079 16888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:41.093259 16888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.24.215.131
I20260810 14:49:41.102475 16888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:38849
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.24.215.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.24.215.131
--webserver_port=0
--tserver_master_addrs=127.24.215.190:34645
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.24.215.131
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:41.103785 16888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:41.105573 16888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:41.119539 16893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:41.119509 16894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:41.119477 16896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:41.120581 16888 server_base.cc:1034] running on GCE node
I20260810 14:49:41.121250 16888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:41.122351 16888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:41.123582 16888 hybrid_clock.cc:648] HybridClock initialized: now 1786373381123528 us; error 41 us; skew 500 ppm
I20260810 14:49:41.123912 16888 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260810 14:49:41.126207 16888 webserver.cc:460] Webserver started at http://127.24.215.131:43567/ using document root <none> and password file <none>
I20260810 14:49:41.126900 16888 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:41.127046 16888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:41.127398 16888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:41.130249 16888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/data/instance:
uuid: "f5b031de5e0f40b2a2ac489095c64f2f"
format_stamp: "Formatted at 2026-08-10 14:49:41 on dist-test-slave-vkjp"
server_key: "8e95a936da545417d3c695d974c13e90"
server_key_iv: "71fafc827b5c83f50bec9ec6fc117b2c"
server_key_version: "encryptionkey@0"
I20260810 14:49:41.131040 16888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance:
uuid: "f5b031de5e0f40b2a2ac489095c64f2f"
format_stamp: "Formatted at 2026-08-10 14:49:41 on dist-test-slave-vkjp"
server_key: "8e95a936da545417d3c695d974c13e90"
server_key_iv: "71fafc827b5c83f50bec9ec6fc117b2c"
server_key_version: "encryptionkey@0"
I20260810 14:49:41.136219 16888 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260810 14:49:41.139755 16905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:41.141142 16888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:49:41.141327 16888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/wal
uuid: "f5b031de5e0f40b2a2ac489095c64f2f"
format_stamp: "Formatted at 2026-08-10 14:49:41 on dist-test-slave-vkjp"
server_key: "8e95a936da545417d3c695d974c13e90"
server_key_iv: "71fafc827b5c83f50bec9ec6fc117b2c"
server_key_version: "encryptionkey@0"
I20260810 14:49:41.141552 16888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:41.154779 16888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:41.155680 16888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:41.155997 16888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:41.156929 16888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260810 14:49:41.158375 16888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260810 14:49:41.158502 16888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:41.158628 16888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260810 14:49:41.158722 16888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:41.194339 16888 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.131:36727
I20260810 14:49:41.194430 17032 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.131:36727 every 8 connection(s)
I20260810 14:49:41.195712 16888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/data/info.pb
I20260810 14:49:41.201857 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 16888
I20260810 14:49:41.202077 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/wal/instance
I20260810 14:49:41.202565 25438 external_mini_cluster.cc:1428] Setting key a4bf831cf07e7e3df9ecbff35eeb14ba
I20260810 14:49:41.208709 16740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.198281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.131:35717 (local address 127.24.215.190:34645)
0810 14:49:41.198468 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:41.198472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:41.199827 (+ 1355us) server_negotiation.cc:407] Connection header received
0810 14:49:41.201673 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:41.201678 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:41.201771 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:41.201886 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:41.203113 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.204142 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.205811 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.206224 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.206354 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.207823 (+ 1469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:41.207828 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:41.207831 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:41.207849 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:41.207907 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:41.207926 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:41.208029 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:41.208260 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 14:49:41.208355 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 14:49:41.209630 17035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.198665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34645 (local address 127.24.215.131:35717)
0810 14:49:41.199652 (+ 987us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:41.199694 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:41.201406 (+ 1712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:41.202127 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:41.202136 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:41.202260 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:49:41.202917 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:41.202935 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.204287 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.204299 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:41.205666 (+ 1367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:41.205679 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.206454 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.206462 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:41.206691 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.207414 (+ 723us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0810 14:49:41.207416 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:49:41.207456 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0810 14:49:41.207586 (+ 130us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:49:41.208033 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:49:41.208046 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:49:41.208057 (+ 11us) client_negotiation.cc:769] Sending connection context
0810 14:49:41.208246 (+ 189us) client_negotiation.cc:240] Negotiation successful
0810 14:49:41.208621 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":339,"threads_started":1}
I20260810 14:49:41.210860 17033 heartbeater.cc:347] Connected to a master server at 127.24.215.190:34645
I20260810 14:49:41.211231 17033 heartbeater.cc:464] Registering TS with master...
I20260810 14:49:41.211979 17033 heartbeater.cc:511] Master 127.24.215.190:34645 requested a full tablet report, sending...
I20260810 14:49:41.213310 16529 ts_manager.cc:194] Registered new tserver with Master: f5b031de5e0f40b2a2ac489095c64f2f (127.24.215.131:36727)
I20260810 14:49:41.214099 16529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.24.215.131:35717
I20260810 14:49:41.217552 25438 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260810 14:49:41.226953 16740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.221547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46076 (local address 127.24.215.190:34645)
0810 14:49:41.221755 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:41.221760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:41.222197 (+ 437us) server_negotiation.cc:407] Connection header received
0810 14:49:41.222315 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:41.222319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:41.222383 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:41.222483 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:41.222990 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.223728 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.224985 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.225373 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.225464 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.226417 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:41.226422 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:41.226425 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:41.226452 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:41.226483 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:41.226495 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:41.226565 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:41.226660 (+ 95us) server_negotiation.cc:299] Negotiation successful
0810 14:49:41.226725 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260810 14:49:41.230804 16529 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:46076:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260810 14:49:41.237834 16529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260810 14:49:41.256600 16738 heartbeater.cc:503] Master 127.24.215.190:34645 was elected leader, sending a full tablet report...
I20260810 14:49:41.261642 17050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.251935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:36727 (local address 127.0.0.1:41070)
0810 14:49:41.252602 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:41.252621 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:41.252731 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:41.253941 (+ 1210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:41.253950 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:41.253993 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:41.254847 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:41.254872 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.257000 (+ 2128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.257005 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:41.259271 (+ 2266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:41.259287 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.260588 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.260595 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:41.260817 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.260870 (+ 53us) client_negotiation.cc:769] Sending connection context
0810 14:49:41.261028 (+ 158us) client_negotiation.cc:240] Negotiation successful
0810 14:49:41.261226 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":547,"mutex_wait_us":133,"spinlock_wait_cycles":1280,"thread_start_us":461,"threads_started":1}
I20260810 14:49:41.262072 17046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.249883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.130:37825 (local address 127.0.0.1:34474)
0810 14:49:41.250642 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:41.250683 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:41.250862 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:41.253946 (+ 3084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:41.253951 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:41.254082 (+ 131us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:41.255276 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:41.255282 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.256639 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.256646 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:41.259312 (+ 2666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:41.259336 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.261233 (+ 1897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.261240 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:41.261410 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.261443 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:49:41.261519 (+ 76us) client_negotiation.cc:240] Negotiation successful
0810 14:49:41.261668 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":146,"spinlock_wait_cycles":135680,"thread_start_us":352,"threads_started":1}
I20260810 14:49:41.262408 17051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.252644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41070 (local address 127.24.215.131:36727)
0810 14:49:41.253414 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:41.253421 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:41.253446 (+ 25us) server_negotiation.cc:407] Connection header received
0810 14:49:41.253532 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:41.253538 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:41.253706 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:41.254406 (+ 700us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:41.255073 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.256821 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.259466 (+ 2645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.260375 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.260588 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.260824 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:41.261078 (+ 254us) server_negotiation.cc:299] Negotiation successful
0810 14:49:41.261269 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":611,"thread_start_us":457,"threads_started":1}
I20260810 14:49:41.263157 17047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.250078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34474 (local address 127.24.215.130:37825)
0810 14:49:41.253274 (+ 3196us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:41.253297 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:41.253330 (+ 33us) server_negotiation.cc:407] Connection header received
0810 14:49:41.253435 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:41.253445 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:41.253689 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:41.254111 (+ 422us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:41.255411 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.256466 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.259569 (+ 3103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.261002 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.262348 (+ 1346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.262576 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:41.262656 (+ 80us) server_negotiation.cc:299] Negotiation successful
0810 14:49:41.262773 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3042,"thread_start_us":2747,"threads_started":1}
I20260810 14:49:41.264071 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.251011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:42239 (local address 127.0.0.1:43742)
0810 14:49:41.251675 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:41.251694 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:41.251810 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:41.253941 (+ 2131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:41.253950 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:41.254036 (+ 86us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:41.255056 (+ 1020us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:41.255068 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.258514 (+ 3446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.258520 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:41.260743 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:41.260760 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.263291 (+ 2531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.263300 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:41.263473 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.263506 (+ 33us) client_negotiation.cc:769] Sending connection context
0810 14:49:41.263589 (+ 83us) client_negotiation.cc:240] Negotiation successful
0810 14:49:41.263715 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":542,"mutex_wait_us":83,"spinlock_wait_cycles":10880,"thread_start_us":693,"threads_started":1}
I20260810 14:49:41.266788 16812 tablet_service.cc:1467] Processing CreateTablet for tablet 5b83dda95bc142d3b16a4aa5d6e03db2 (DEFAULT_TABLE table=test-table [id=31117a32ca7347679fad3453f8d2468c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:41.266394 16961 tablet_service.cc:1467] Processing CreateTablet for tablet 5b83dda95bc142d3b16a4aa5d6e03db2 (DEFAULT_TABLE table=test-table [id=31117a32ca7347679fad3453f8d2468c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:41.267936 17049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.251510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43742 (local address 127.24.215.129:42239)
0810 14:49:41.252357 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:41.253314 (+ 957us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:41.253346 (+ 32us) server_negotiation.cc:407] Connection header received
0810 14:49:41.253446 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:41.253453 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:41.253697 (+ 244us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:41.254206 (+ 509us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:41.255267 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.258320 (+ 3053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.260938 (+ 2618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.262396 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.263313 (+ 917us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.267253 (+ 3940us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:41.267380 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 14:49:41.267556 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":658,"thread_start_us":546,"threads_started":1}
I20260810 14:49:41.268281 16812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b83dda95bc142d3b16a4aa5d6e03db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:41.268278 16961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b83dda95bc142d3b16a4aa5d6e03db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:41.270741 16666 tablet_service.cc:1467] Processing CreateTablet for tablet 5b83dda95bc142d3b16a4aa5d6e03db2 (DEFAULT_TABLE table=test-table [id=31117a32ca7347679fad3453f8d2468c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260810 14:49:41.271934 16666 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b83dda95bc142d3b16a4aa5d6e03db2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:41.283434 17053 tablet_bootstrap.cc:492] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10: Bootstrap starting.
I20260810 14:49:41.284271 17052 tablet_bootstrap.cc:492] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5: Bootstrap starting.
I20260810 14:49:41.285776 17054 tablet_bootstrap.cc:492] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f: Bootstrap starting.
I20260810 14:49:41.288415 17052 tablet_bootstrap.cc:654] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:41.288654 17054 tablet_bootstrap.cc:654] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:41.288640 17053 tablet_bootstrap.cc:654] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:41.289956 17054 log.cc:824] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:41.290146 17052 log.cc:824] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:41.290146 17053 log.cc:824] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:41.292688 17054 tablet_bootstrap.cc:492] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f: No bootstrap required, opened a new log
I20260810 14:49:41.293009 17054 ts_tablet_manager.cc:1397] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.005s
I20260810 14:49:41.295466 17052 tablet_bootstrap.cc:492] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5: No bootstrap required, opened a new log
I20260810 14:49:41.295473 17053 tablet_bootstrap.cc:492] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10: No bootstrap required, opened a new log
I20260810 14:49:41.295740 17053 ts_tablet_manager.cc:1397] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10: Time spent bootstrapping tablet: real 0.013s user 0.008s sys 0.002s
I20260810 14:49:41.295740 17052 ts_tablet_manager.cc:1397] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.005s
I20260810 14:49:41.300587 17054 raft_consensus.cc:348] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7576b90769842ccb5ef1e1189515c10" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37825 } } peers { permanent_uuid: "e7eb528413cb40acb7123a5d3ab372c5" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42239 } } peers { permanent_uuid: "f5b031de5e0f40b2a2ac489095c64f2f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 36727 } }
I20260810 14:49:41.301002 17054 raft_consensus.cc:374] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:41.301146 17054 raft_consensus.cc:729] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5b031de5e0f40b2a2ac489095c64f2f, State: Initialized, Role: FOLLOWER
I20260810 14:49:41.301810 17054 consensus_queue.cc:260] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7576b90769842ccb5ef1e1189515c10" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37825 } } peers { permanent_uuid: "e7eb528413cb40acb7123a5d3ab372c5" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42239 } } peers { permanent_uuid: "f5b031de5e0f40b2a2ac489095c64f2f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 36727 } }
I20260810 14:49:41.303299 17033 heartbeater.cc:503] Master 127.24.215.190:34645 was elected leader, sending a full tablet report...
I20260810 14:49:41.303696 17054 ts_tablet_manager.cc:1428] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f: Time spent starting tablet: real 0.011s user 0.010s sys 0.000s
I20260810 14:49:41.306414 17052 raft_consensus.cc:348] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7576b90769842ccb5ef1e1189515c10" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37825 } } peers { permanent_uuid: "e7eb528413cb40acb7123a5d3ab372c5" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42239 } } peers { permanent_uuid: "f5b031de5e0f40b2a2ac489095c64f2f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 36727 } }
I20260810 14:49:41.306972 17052 raft_consensus.cc:374] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:41.307188 17052 raft_consensus.cc:729] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e7eb528413cb40acb7123a5d3ab372c5, State: Initialized, Role: FOLLOWER
I20260810 14:49:41.307823 17052 consensus_queue.cc:260] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7576b90769842ccb5ef1e1189515c10" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37825 } } peers { permanent_uuid: "e7eb528413cb40acb7123a5d3ab372c5" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42239 } } peers { permanent_uuid: "f5b031de5e0f40b2a2ac489095c64f2f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 36727 } }
I20260810 14:49:41.308131 17053 raft_consensus.cc:348] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7576b90769842ccb5ef1e1189515c10" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37825 } } peers { permanent_uuid: "e7eb528413cb40acb7123a5d3ab372c5" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42239 } } peers { permanent_uuid: "f5b031de5e0f40b2a2ac489095c64f2f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 36727 } }
I20260810 14:49:41.308691 17053 raft_consensus.cc:374] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:41.309762 17052 ts_tablet_manager.cc:1428] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260810 14:49:41.308838 17053 raft_consensus.cc:729] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7576b90769842ccb5ef1e1189515c10, State: Initialized, Role: FOLLOWER
I20260810 14:49:41.310626 17053 consensus_queue.cc:260] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7576b90769842ccb5ef1e1189515c10" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37825 } } peers { permanent_uuid: "e7eb528413cb40acb7123a5d3ab372c5" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42239 } } peers { permanent_uuid: "f5b031de5e0f40b2a2ac489095c64f2f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 36727 } }
I20260810 14:49:41.311793 16885 heartbeater.cc:503] Master 127.24.215.190:34645 was elected leader, sending a full tablet report...
I20260810 14:49:41.312459 17053 ts_tablet_manager.cc:1428] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10: Time spent starting tablet: real 0.017s user 0.011s sys 0.002s
I20260810 14:49:41.357064 17061 raft_consensus.cc:482] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260810 14:49:41.357348 17061 raft_consensus.cc:504] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7576b90769842ccb5ef1e1189515c10" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37825 } } peers { permanent_uuid: "e7eb528413cb40acb7123a5d3ab372c5" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42239 } } peers { permanent_uuid: "f5b031de5e0f40b2a2ac489095c64f2f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 36727 } }
I20260810 14:49:41.359023 17061 leader_election.cc:288] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e7eb528413cb40acb7123a5d3ab372c5 (127.24.215.129:42239), f5b031de5e0f40b2a2ac489095c64f2f (127.24.215.131:36727)
I20260810 14:49:41.366242 17051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.359662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:40359 (local address 127.24.215.131:36727)
0810 14:49:41.359906 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:41.359915 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:41.360630 (+ 715us) server_negotiation.cc:407] Connection header received
0810 14:49:41.360818 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:41.360821 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:41.360877 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:41.360988 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:41.361702 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.362439 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.364471 (+ 2032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.365249 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.365418 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.365504 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:41.365846 (+ 342us) server_negotiation.cc:299] Negotiation successful
0810 14:49:41.365979 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 14:49:41.366242 17049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.359376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.24.215.130:40091 (local address 127.24.215.129:42239)
0810 14:49:41.359588 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:41.359594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:41.360555 (+ 961us) server_negotiation.cc:407] Connection header received
0810 14:49:41.360760 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:41.360765 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:41.360858 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:41.360928 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0810 14:49:41.361628 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.362343 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.364431 (+ 2088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.365243 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.365454 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.365520 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:41.365846 (+ 326us) server_negotiation.cc:299] Negotiation successful
0810 14:49:41.365983 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260810 14:49:41.366577 17063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.359589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.131:36727 (local address 127.24.215.130:40359)
0810 14:49:41.360475 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:41.360518 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:41.360681 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:41.361036 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:41.361042 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:41.361083 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:41.361508 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:41.361518 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.362581 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.362584 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:41.364222 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:41.364234 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.365475 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.365482 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:41.365683 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.365708 (+ 25us) client_negotiation.cc:769] Sending connection context
0810 14:49:41.365815 (+ 107us) client_negotiation.cc:240] Negotiation successful
0810 14:49:41.365992 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":783,"mutex_wait_us":90,"thread_start_us":529,"threads_started":1}
I20260810 14:49:41.366602 17062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.359517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:42239 (local address 127.24.215.130:40091)
0810 14:49:41.360426 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:41.360442 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:49:41.360594 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:49:41.360976 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:49:41.360980 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:49:41.361005 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0810 14:49:41.361461 (+ 456us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:41.361474 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.362477 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.362494 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:41.364218 (+ 1724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:49:41.364230 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.365499 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.365506 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:49:41.365688 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.365711 (+ 23us) client_negotiation.cc:769] Sending connection context
0810 14:49:41.365814 (+ 103us) client_negotiation.cc:240] Negotiation successful
0810 14:49:41.365987 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":737,"mutex_wait_us":141,"thread_start_us":512,"threads_started":1}
I20260810 14:49:41.367660 16688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b83dda95bc142d3b16a4aa5d6e03db2" candidate_uuid: "f7576b90769842ccb5ef1e1189515c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7eb528413cb40acb7123a5d3ab372c5" is_pre_election: true
I20260810 14:49:41.367655 16982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b83dda95bc142d3b16a4aa5d6e03db2" candidate_uuid: "f7576b90769842ccb5ef1e1189515c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5b031de5e0f40b2a2ac489095c64f2f" is_pre_election: true
I20260810 14:49:41.368175 16688 raft_consensus.cc:2454] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7576b90769842ccb5ef1e1189515c10 in term 0.
I20260810 14:49:41.368192 16982 raft_consensus.cc:2454] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f7576b90769842ccb5ef1e1189515c10 in term 0.
I20260810 14:49:41.368808 16760 leader_election.cc:302] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e7eb528413cb40acb7123a5d3ab372c5, f7576b90769842ccb5ef1e1189515c10; no voters:
I20260810 14:49:41.369220 17061 raft_consensus.cc:2781] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260810 14:49:41.369343 17061 raft_consensus.cc:482] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260810 14:49:41.369414 17061 raft_consensus.cc:3037] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:41.371654 17061 raft_consensus.cc:504] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7576b90769842ccb5ef1e1189515c10" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37825 } } peers { permanent_uuid: "e7eb528413cb40acb7123a5d3ab372c5" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42239 } } peers { permanent_uuid: "f5b031de5e0f40b2a2ac489095c64f2f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 36727 } }
I20260810 14:49:41.372362 17061 leader_election.cc:288] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [CANDIDATE]: Term 1 election: Requested vote from peers e7eb528413cb40acb7123a5d3ab372c5 (127.24.215.129:42239), f5b031de5e0f40b2a2ac489095c64f2f (127.24.215.131:36727)
I20260810 14:49:41.372838 16688 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b83dda95bc142d3b16a4aa5d6e03db2" candidate_uuid: "f7576b90769842ccb5ef1e1189515c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e7eb528413cb40acb7123a5d3ab372c5"
I20260810 14:49:41.372898 16982 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b83dda95bc142d3b16a4aa5d6e03db2" candidate_uuid: "f7576b90769842ccb5ef1e1189515c10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5b031de5e0f40b2a2ac489095c64f2f"
I20260810 14:49:41.373070 16688 raft_consensus.cc:3037] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:41.373080 16982 raft_consensus.cc:3037] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:41.376426 16688 raft_consensus.cc:2454] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7576b90769842ccb5ef1e1189515c10 in term 1.
I20260810 14:49:41.376426 16982 raft_consensus.cc:2454] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f7576b90769842ccb5ef1e1189515c10 in term 1.
I20260810 14:49:41.376961 16761 leader_election.cc:302] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: e7eb528413cb40acb7123a5d3ab372c5, f5b031de5e0f40b2a2ac489095c64f2f, f7576b90769842ccb5ef1e1189515c10; no voters:
I20260810 14:49:41.377425 17061 raft_consensus.cc:2781] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:41.377907 17061 raft_consensus.cc:686] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [term 1 LEADER]: Becoming Leader. State: Replica: f7576b90769842ccb5ef1e1189515c10, State: Running, Role: LEADER
I20260810 14:49:41.378381 17061 consensus_queue.cc:237] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7576b90769842ccb5ef1e1189515c10" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37825 } } peers { permanent_uuid: "e7eb528413cb40acb7123a5d3ab372c5" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42239 } } peers { permanent_uuid: "f5b031de5e0f40b2a2ac489095c64f2f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 36727 } }
I20260810 14:49:41.383134 16528 catalog_manager.cc:5495] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 reported cstate change: term changed from 0 to 1, leader changed from <none> to f7576b90769842ccb5ef1e1189515c10 (127.24.215.130). New cstate: current_term: 1 leader_uuid: "f7576b90769842ccb5ef1e1189515c10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f7576b90769842ccb5ef1e1189515c10" member_type: VOTER last_known_addr { host: "127.24.215.130" port: 37825 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e7eb528413cb40acb7123a5d3ab372c5" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42239 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f5b031de5e0f40b2a2ac489095c64f2f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 36727 } health_report { overall_health: UNKNOWN } } }
I20260810 14:49:41.415205 17047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.409633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34478 (local address 127.24.215.130:37825)
0810 14:49:41.409832 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:41.409840 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:41.409948 (+ 108us) server_negotiation.cc:407] Connection header received
0810 14:49:41.410209 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:41.410215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:41.410310 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:41.410476 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:49:41.410850 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.411589 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.413010 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.413431 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.413567 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.414076 (+ 509us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:49:41.414570 (+ 494us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:49:41.414739 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:41.414822 (+ 83us) server_negotiation.cc:299] Negotiation successful
0810 14:49:41.414901 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260810 14:49:41.423527 16982 raft_consensus.cc:1261] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f [term 1 FOLLOWER]: Refusing update from remote peer f7576b90769842ccb5ef1e1189515c10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:49:41.423522 16688 raft_consensus.cc:1261] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5 [term 1 FOLLOWER]: Refusing update from remote peer f7576b90769842ccb5ef1e1189515c10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:49:41.424491 17064 consensus_queue.cc:1035] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5b031de5e0f40b2a2ac489095c64f2f" member_type: VOTER last_known_addr { host: "127.24.215.131" port: 36727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:49:41.424829 17061 consensus_queue.cc:1035] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e7eb528413cb40acb7123a5d3ab372c5" member_type: VOTER last_known_addr { host: "127.24.215.129" port: 42239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:49:41.451751 16528 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:46076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:49:41.452054 16528 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:46076:
table { table_name: "test-table" } modify_external_catalogs: true
I20260810 14:49:41.456530 16528 catalog_manager.cc:5782] T 00000000000000000000000000000000 P de601544fbdc41c8b97f61008153ad4f: Sending DeleteTablet for 3 replicas of tablet 5b83dda95bc142d3b16a4aa5d6e03db2
I20260810 14:49:41.457783 16812 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b83dda95bc142d3b16a4aa5d6e03db2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:41 UTC) from {username='slave'} at 127.0.0.1:34474
I20260810 14:49:41.458015 16961 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b83dda95bc142d3b16a4aa5d6e03db2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:41 UTC) from {username='slave'} at 127.0.0.1:41070
I20260810 14:49:41.458587 16666 tablet_service.cc:1514] Processing DeleteTablet for tablet 5b83dda95bc142d3b16a4aa5d6e03db2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-10 14:49:41 UTC) from {username='slave'} at 127.0.0.1:43742
I20260810 14:49:41.458776 17078 tablet_replica.cc:330] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10: stopping tablet replica
I20260810 14:49:41.459271 17078 raft_consensus.cc:2229] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [term 1 LEADER]: Raft consensus shutting down.
I20260810 14:49:41.460026 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 16594
I20260810 14:49:41.463378 17078 raft_consensus.cc:2258] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:41.466204 17078 ts_tablet_manager.cc:1905] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:41.469518 17080 tablet_replica.cc:330] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5: stopping tablet replica
I20260810 14:49:41.470083 17080 raft_consensus.cc:2229] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:49:41.470633 17080 raft_consensus.cc:2258] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:41.473507 17049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.461137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43758 (local address 127.24.215.129:42239)
0810 14:49:41.461372 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:41.461381 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:41.464281 (+ 2900us) server_negotiation.cc:407] Connection header received
0810 14:49:41.464548 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:41.464553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:41.464678 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:41.464764 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:41.465458 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.466663 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.468689 (+ 2026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.469428 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.469541 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.471040 (+ 1499us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:41.471052 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:41.471069 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:41.471873 (+ 804us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:41.471944 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:41.471978 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:41.472063 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:41.473022 (+ 959us) server_negotiation.cc:299] Negotiation successful
0810 14:49:41.473139 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260810 14:49:41.474387 17080 ts_tablet_manager.cc:1905] T 5b83dda95bc142d3b16a4aa5d6e03db2 P e7eb528413cb40acb7123a5d3ab372c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:41.474681 16731 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:41.479691 17079 tablet_replica.cc:330] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f: stopping tablet replica
I20260810 14:49:41.480181 17079 raft_consensus.cc:2229] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260810 14:49:41.480561 17078 ts_tablet_manager.cc:1918] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:49:41.480765 17079 raft_consensus.cc:2258] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260810 14:49:41.480777 17078 log.cc:1196] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-1/wal/wals/5b83dda95bc142d3b16a4aa5d6e03db2
I20260810 14:49:41.481662 17078 ts_tablet_manager.cc:1939] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f7576b90769842ccb5ef1e1189515c10: Deleting consensus metadata
I20260810 14:49:41.482755 17079 ts_tablet_manager.cc:1905] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260810 14:49:41.483767 16512 catalog_manager.cc:4841] TS f7576b90769842ccb5ef1e1189515c10 (127.24.215.130:37825): tablet 5b83dda95bc142d3b16a4aa5d6e03db2 (table test-table [id=31117a32ca7347679fad3453f8d2468c]) successfully deleted
I20260810 14:49:41.489938 17079 ts_tablet_manager.cc:1918] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260810 14:49:41.490098 17079 log.cc:1196] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f: Deleting WAL directory at /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786373051600012-25438-0/minicluster-data/ts-2/wal/wals/5b83dda95bc142d3b16a4aa5d6e03db2
I20260810 14:49:41.490622 17079 ts_tablet_manager.cc:1939] T 5b83dda95bc142d3b16a4aa5d6e03db2 P f5b031de5e0f40b2a2ac489095c64f2f: Deleting consensus metadata
I20260810 14:49:41.492544 16513 catalog_manager.cc:4841] TS f5b031de5e0f40b2a2ac489095c64f2f (127.24.215.131:36727): tablet 5b83dda95bc142d3b16a4aa5d6e03db2 (table test-table [id=31117a32ca7347679fad3453f8d2468c]) successfully deleted
I20260810 14:49:41.567221 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 16594
W20260810 14:49:41.592447 16512 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.24.215.129:42239 (error 108)
I20260810 14:49:41.593364 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 16742
I20260810 14:49:41.595278 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.594628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:42239 (local address 127.0.0.1:43762)
0810 14:49:41.594897 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:41.595037 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:42239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147}
W20260810 14:49:41.595556 16512 catalog_manager.cc:4584] TS e7eb528413cb40acb7123a5d3ab372c5 (127.24.215.129:42239): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5b83dda95bc142d3b16a4aa5d6e03db2: Network error: Client connection negotiation failed: client connection to 127.24.215.129:42239: connect: Connection refused (error 111)
I20260810 14:49:41.602138 17047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.594772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34492 (local address 127.24.215.130:37825)
0810 14:49:41.594934 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:41.594940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:41.595117 (+ 177us) server_negotiation.cc:407] Connection header received
0810 14:49:41.595319 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:41.595327 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:41.595469 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:41.595593 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:41.596052 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.596829 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.598693 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.599327 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.599428 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.600550 (+ 1122us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:41.600558 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:41.600572 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:41.601275 (+ 703us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:41.601347 (+ 72us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:41.601371 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:41.601449 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:41.601664 (+ 215us) server_negotiation.cc:299] Negotiation successful
0810 14:49:41.601756 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 14:49:41.602918 16880 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:41.649248 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.648687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:42239 (local address 127.0.0.1:43772)
0810 14:49:41.648921 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:41.649043 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:42239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 14:49:41.650326 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.649925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:42239 (local address 127.0.0.1:43778)
0810 14:49:41.650097 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:41.650197 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:42239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 14:49:41.710781 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.710219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:42239 (local address 127.0.0.1:43794)
0810 14:49:41.710473 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:41.710564 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:42239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260810 14:49:41.711946 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.711601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:42239 (local address 127.0.0.1:43796)
0810 14:49:41.711755 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:41.711830 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:42239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260810 14:49:41.715152 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 16742
I20260810 14:49:41.739178 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 16888
I20260810 14:49:41.746279 17051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.739963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41086 (local address 127.24.215.131:36727)
0810 14:49:41.740160 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:41.740169 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:41.740207 (+ 38us) server_negotiation.cc:407] Connection header received
0810 14:49:41.740451 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:41.740461 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:41.740560 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:41.740673 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:41.741115 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.741901 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.743241 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:41.743633 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:41.743773 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:41.744822 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:41.744831 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:41.744846 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:41.745482 (+ 636us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:41.745530 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:41.745557 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:41.745680 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:41.745831 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 14:49:41.745940 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260810 14:49:41.746883 17028 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:41.792341 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.791762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:42239 (local address 127.0.0.1:43802)
0810 14:49:41.792031 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:41.792161 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:42239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260810 14:49:41.793545 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.793075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:42239 (local address 127.0.0.1:43808)
0810 14:49:41.793303 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:41.793392 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:42239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260810 14:49:41.838573 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 16888
I20260810 14:49:41.859681 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 16493
I20260810 14:49:41.860839 16564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:41.971107 17048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:41.970366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.129:42239 (local address 127.0.0.1:43814)
0810 14:49:41.970613 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0810 14:49:41.970791 (+ 178us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.129:42239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260810 14:49:41.971311 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 16493
2026-08-10T14:49:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2854 ms)
[----------] 6 tests from EncryptionPolicyTest (23462 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:49:42 dist-test-slave-vkjp krb5kdc[17091](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:49:42 dist-test-slave-vkjp krb5kdc[17091](info): set up 2 sockets
Aug 10 14:49:42 dist-test-slave-vkjp krb5kdc[17091](info): commencing operation
krb5kdc: starting...
W20260810 14:49:43.181167 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:49:43 dist-test-slave-vkjp krb5kdc[17091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:49:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:49:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:49:43.482141 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39343
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37685
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39343
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:43.827910 17105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:43.828212 17105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:43.828402 17105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:43.835760 17105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:49:43.835866 17105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:43.835971 17105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:43.836090 17105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:49:43.836143 17105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:49:43.846833 17105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37685
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39343
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:43.848212 17105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:43.849658 17105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:43.861544 17111 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:43.861603 17112 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:43.861603 17114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:43.862879 17105 server_base.cc:1034] running on GCE node
I20260810 14:49:43.863775 17105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:43.865203 17105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:43.866436 17105 hybrid_clock.cc:648] HybridClock initialized: now 1786373383866392 us; error 58 us; skew 500 ppm
I20260810 14:49:43.866796 17105 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:43 dist-test-slave-vkjp krb5kdc[17091](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373383, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:43.869949 17105 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:49:43.871500 17105 webserver.cc:460] Webserver started at http://127.0.0.1:38327/ using document root <none> and password file <none>
I20260810 14:49:43.872190 17105 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:43.872340 17105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:43.872655 17105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:43.875738 17105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "7f881f3bb7f84085aee4eda669d4f775"
format_stamp: "Formatted at 2026-08-10 14:49:43 on dist-test-slave-vkjp"
server_key: "4f5bd701f2ca09557b92533d88cd483d"
server_key_iv: "163fc374794620a49cbfc904b778a83a"
server_key_version: "encryptionkey@0"
I20260810 14:49:43.876540 17105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "7f881f3bb7f84085aee4eda669d4f775"
format_stamp: "Formatted at 2026-08-10 14:49:43 on dist-test-slave-vkjp"
server_key: "4f5bd701f2ca09557b92533d88cd483d"
server_key_iv: "163fc374794620a49cbfc904b778a83a"
server_key_version: "encryptionkey@0"
I20260810 14:49:43.881539 17105 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:49:43.885069 17122 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:43.886368 17105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:49:43.886557 17105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "7f881f3bb7f84085aee4eda669d4f775"
format_stamp: "Formatted at 2026-08-10 14:49:43 on dist-test-slave-vkjp"
server_key: "4f5bd701f2ca09557b92533d88cd483d"
server_key_iv: "163fc374794620a49cbfc904b778a83a"
server_key_version: "encryptionkey@0"
I20260810 14:49:43.886766 17105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:43.902388 17105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:43.906312 17105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:43.906602 17105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:43.925515 17105 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39343
I20260810 14:49:43.925539 17176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39343 every 8 connection(s)
I20260810 14:49:43.926918 17105 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:49:43.930711 17177 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:43.931011 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 17105
I20260810 14:49:43.931241 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:49:43.931921 25438 external_mini_cluster.cc:1428] Setting key 6571fd2bd8e0237f51b87917a2e76217
I20260810 14:49:43.945000 17177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775: Bootstrap starting.
Aug 10 14:49:43 dist-test-slave-vkjp krb5kdc[17091](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373383, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 14:49:43.949167 17177 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:43.950291 17177 log.cc:824] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:43.953814 17177 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775: No bootstrap required, opened a new log
I20260810 14:49:43.957459 17180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:43.933864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44633 (local address 127.0.0.1:39343)
0810 14:49:43.934733 (+ 869us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:43.934744 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:43.934785 (+ 41us) server_negotiation.cc:407] Connection header received
0810 14:49:43.936855 (+ 2070us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:43.936880 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:43.937273 (+ 393us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:43.937605 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:43.939512 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:43.940874 (+ 1362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:43.942808 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:43.943542 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:43.943680 (+ 138us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:43.948902 (+ 5222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:43.948911 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:43.948925 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:43.948963 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:43.952647 (+ 3684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:43.953631 (+ 984us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:43.953637 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:43.953643 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:43.953804 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:43.954372 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:43.954377 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:43.954379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:43.954891 (+ 512us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:43.955395 (+ 504us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:43.955989 (+ 594us) server_negotiation.cc:299] Negotiation successful
0810 14:49:43.956344 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":383,"threads_started":1}
I20260810 14:49:43.962240 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:49:43.962639 17177 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f881f3bb7f84085aee4eda669d4f775" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39343 } }
I20260810 14:49:43.963006 17177 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:43.963130 17177 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f881f3bb7f84085aee4eda669d4f775, State: Initialized, Role: FOLLOWER
I20260810 14:49:43.963686 17177 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f881f3bb7f84085aee4eda669d4f775" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39343 } }
I20260810 14:49:43.963943 17177 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:49:43.964093 17177 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:49:43.964277 17177 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:43.966686 17177 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f881f3bb7f84085aee4eda669d4f775" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39343 } }
I20260810 14:49:43.967496 17177 leader_election.cc:302] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7f881f3bb7f84085aee4eda669d4f775; no voters:
I20260810 14:49:43.968235 17177 leader_election.cc:288] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:49:43.968364 17187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:43.970479 17177 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:49:43.971982 17187 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [term 1 LEADER]: Becoming Leader. State: Replica: 7f881f3bb7f84085aee4eda669d4f775, State: Running, Role: LEADER
I20260810 14:49:43.972687 17187 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f881f3bb7f84085aee4eda669d4f775" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39343 } }
I20260810 14:49:43.975647 17188 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7f881f3bb7f84085aee4eda669d4f775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f881f3bb7f84085aee4eda669d4f775" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39343 } } }
I20260810 14:49:43.976073 17188 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:43.977839 17189 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7f881f3bb7f84085aee4eda669d4f775. Latest consensus state: current_term: 1 leader_uuid: "7f881f3bb7f84085aee4eda669d4f775" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7f881f3bb7f84085aee4eda669d4f775" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39343 } } }
I20260810 14:49:43.978655 17189 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:43.980123 17196 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:49:43.985812 17196 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:49:43.987545 17180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:43.967560 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53353 (local address 127.0.0.1:39343)
0810 14:49:43.967733 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:43.967741 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:43.968316 (+ 575us) server_negotiation.cc:407] Connection header received
0810 14:49:43.968551 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:43.968557 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:43.968651 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:43.968736 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:43.971287 (+ 2551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:43.972624 (+ 1337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:43.974490 (+ 1866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:43.975266 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:43.975407 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:43.979288 (+ 3881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:43.979298 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:43.979305 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:43.979356 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:43.983214 (+ 3858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:43.984252 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:43.984258 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:43.984262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:43.984428 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:43.985046 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:43.985051 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:43.985056 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:43.985491 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:43.985796 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:43.986976 (+ 1180us) server_negotiation.cc:299] Negotiation successful
0810 14:49:43.987189 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260810 14:49:43.995855 17196 catalog_manager.cc:1347] Generated new cluster ID: 1b774fc1db5644099d1ede48e159136d
I20260810 14:49:43.995970 17196 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:49:44.014202 17196 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:49:44.015626 17196 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:49:44.024879 17196 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7f881f3bb7f84085aee4eda669d4f775: Generated new TSK 0
I20260810 14:49:44.026054 17196 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:49:44.041508 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 17105
I20260810 14:49:44.042914 17172 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:44.135504 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 17105
W20260810 14:49:44.157428 17088 connection.cc:535] client connection to 127.0.0.1:39343 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:39343): Connection reset by peer (error 104)
2026-08-10T14:49:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2154 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:49:44 dist-test-slave-vkjp krb5kdc[17211](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:49:44 dist-test-slave-vkjp krb5kdc[17211](info): set up 2 sockets
Aug 10 14:49:44 dist-test-slave-vkjp krb5kdc[17211](info): commencing operation
krb5kdc: starting...
W20260810 14:49:45.336397 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:49:45 dist-test-slave-vkjp krb5kdc[17211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373385, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:49:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:49:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:49:45.643730 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41271
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40635
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41271
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:45.987753 17228 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:45.988044 17228 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:45.988135 17228 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:45.995240 17228 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:49:45.995350 17228 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:45.995397 17228 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:45.995451 17228 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:49:45.995496 17228 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:49:46.005380 17228 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40635
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41271
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41271
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:46.006718 17228 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:46.008081 17228 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:46.020129 17233 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:46.020153 17236 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:46.020059 17234 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:46.021109 17228 server_base.cc:1034] running on GCE node
I20260810 14:49:46.022008 17228 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:46.023588 17228 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:46.024834 17228 hybrid_clock.cc:648] HybridClock initialized: now 1786373386024782 us; error 63 us; skew 500 ppm
I20260810 14:49:46.025210 17228 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:46 dist-test-slave-vkjp krb5kdc[17211](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373386, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:46.028534 17228 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:49:46.030114 17228 webserver.cc:460] Webserver started at http://127.0.0.1:34849/ using document root <none> and password file <none>
I20260810 14:49:46.030809 17228 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:46.030978 17228 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:46.031352 17228 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:46.034425 17228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "25138dea3ffc45b99d7fd09f8d3158a3"
format_stamp: "Formatted at 2026-08-10 14:49:46 on dist-test-slave-vkjp"
server_key: "f79f225fd06963e5f7a00c86f33c4962"
server_key_iv: "6b378d3bec0be805541424116ac8e478"
server_key_version: "encryptionkey@0"
I20260810 14:49:46.035295 17228 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "25138dea3ffc45b99d7fd09f8d3158a3"
format_stamp: "Formatted at 2026-08-10 14:49:46 on dist-test-slave-vkjp"
server_key: "f79f225fd06963e5f7a00c86f33c4962"
server_key_iv: "6b378d3bec0be805541424116ac8e478"
server_key_version: "encryptionkey@0"
I20260810 14:49:46.040547 17228 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 14:49:46.044067 17245 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:46.045372 17228 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:49:46.045570 17228 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "25138dea3ffc45b99d7fd09f8d3158a3"
format_stamp: "Formatted at 2026-08-10 14:49:46 on dist-test-slave-vkjp"
server_key: "f79f225fd06963e5f7a00c86f33c4962"
server_key_iv: "6b378d3bec0be805541424116ac8e478"
server_key_version: "encryptionkey@0"
I20260810 14:49:46.045773 17228 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:46.059603 17228 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:46.063769 17228 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:46.064049 17228 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:46.083101 17228 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41271
I20260810 14:49:46.083200 17298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41271 every 8 connection(s)
I20260810 14:49:46.084504 17228 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:49:46.088449 17299 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:46.094388 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 17228
I20260810 14:49:46.094592 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:49:46.095191 25438 external_mini_cluster.cc:1428] Setting key ddb50875fa4349cfdd8a26acd9166348
I20260810 14:49:46.100135 17299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3: Bootstrap starting.
I20260810 14:49:46.103608 17299 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:46.104844 17299 log.cc:824] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:46.108176 17299 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3: No bootstrap required, opened a new log
I20260810 14:49:46.109903 17302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:46.097991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48553 (local address 127.0.0.1:41271)
0810 14:49:46.098784 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:46.098795 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:46.098836 (+ 41us) server_negotiation.cc:407] Connection header received
0810 14:49:46.100600 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:46.100626 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:46.100984 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:46.101294 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:46.101957 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:46.103298 (+ 1341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:46.105183 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:46.106287 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:46.106407 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:46.107698 (+ 1291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:46.107705 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:46.107718 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:46.108212 (+ 494us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:46.108324 (+ 112us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:46.108350 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:46.108470 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:46.108763 (+ 293us) server_negotiation.cc:299] Negotiation successful
0810 14:49:46.108946 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":365,"threads_started":1}
I20260810 14:49:46.113608 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:49:46.117372 17299 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25138dea3ffc45b99d7fd09f8d3158a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } }
I20260810 14:49:46.117866 17299 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:46.117980 17299 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25138dea3ffc45b99d7fd09f8d3158a3, State: Initialized, Role: FOLLOWER
I20260810 14:49:46.118626 17299 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25138dea3ffc45b99d7fd09f8d3158a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } }
I20260810 14:49:46.118954 17299 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:49:46.119196 17299 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:49:46.119416 17299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:46.122400 17299 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25138dea3ffc45b99d7fd09f8d3158a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } }
I20260810 14:49:46.122906 17299 leader_election.cc:302] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25138dea3ffc45b99d7fd09f8d3158a3; no voters:
I20260810 14:49:46.123579 17299 leader_election.cc:288] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:49:46.123683 17309 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:46.124398 17309 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [term 1 LEADER]: Becoming Leader. State: Replica: 25138dea3ffc45b99d7fd09f8d3158a3, State: Running, Role: LEADER
I20260810 14:49:46.125362 17309 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25138dea3ffc45b99d7fd09f8d3158a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } }
I20260810 14:49:46.126371 17302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:46.118979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38247 (local address 127.0.0.1:41271)
0810 14:49:46.119176 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:46.119187 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:46.119639 (+ 452us) server_negotiation.cc:407] Connection header received
0810 14:49:46.119782 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:46.119790 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:46.119876 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:46.119999 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:46.120435 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:46.121598 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:46.122933 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:46.123706 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:46.123846 (+ 140us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:46.125200 (+ 1354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:46.125209 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:46.125217 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:46.125245 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:46.125329 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:46.125358 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:46.125502 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:46.125845 (+ 343us) server_negotiation.cc:299] Negotiation successful
0810 14:49:46.125914 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260810 14:49:46.127851 17309 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25138dea3ffc45b99d7fd09f8d3158a3. Latest consensus state: current_term: 1 leader_uuid: "25138dea3ffc45b99d7fd09f8d3158a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25138dea3ffc45b99d7fd09f8d3158a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } } }
I20260810 14:49:46.128294 17309 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:46.129135 17311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25138dea3ffc45b99d7fd09f8d3158a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25138dea3ffc45b99d7fd09f8d3158a3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41271 } } }
I20260810 14:49:46.129954 17311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:46.132645 17299 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:49:46.132846 17313 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:49:46.138935 17313 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:49:46.148741 17313 catalog_manager.cc:1347] Generated new cluster ID: 29bc9fda0a3d4c7c9ef4d0ae6fa1e8d2
I20260810 14:49:46.148871 17313 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:49:46.162410 17313 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:49:46.164219 17313 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:49:46.171594 17313 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25138dea3ffc45b99d7fd09f8d3158a3: Generated new TSK 0
I20260810 14:49:46.172590 17313 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:49:46.188716 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 17228
W20260810 14:49:46.190505 17294 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48553
W20260810 14:49:46.191347 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 17228: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:49:46.191521 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 17228
W20260810 14:49:46.214876 17209 connection.cc:535] client connection to 127.0.0.1:41271 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41271): Connection reset by peer (error 104)
2026-08-10T14:49:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2057 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:49:46 dist-test-slave-vkjp krb5kdc[17331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:49:46 dist-test-slave-vkjp krb5kdc[17331](info): set up 2 sockets
Aug 10 14:49:46 dist-test-slave-vkjp krb5kdc[17331](info): commencing operation
krb5kdc: starting...
W20260810 14:49:47.400481 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:49:47 dist-test-slave-vkjp krb5kdc[17331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373387, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:49:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:49:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:49:47.700362 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37163
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36921
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37163
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:48.041733 17346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:48.042091 17346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:48.042253 17346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:48.049449 17346 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:49:48.049624 17346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:48.049732 17346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:48.049841 17346 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:49:48.049925 17346 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:49:48.059824 17346 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36921
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37163
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37163
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:48.061230 17346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:48.062539 17346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:48.074584 17353 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:48.074666 17354 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:48.074620 17356 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:48.075789 17346 server_base.cc:1034] running on GCE node
I20260810 14:49:48.076658 17346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:48.078074 17346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:48.079293 17346 hybrid_clock.cc:648] HybridClock initialized: now 1786373388079246 us; error 51 us; skew 500 ppm
I20260810 14:49:48.079640 17346 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:48 dist-test-slave-vkjp krb5kdc[17331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373388, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:48.082999 17346 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:49:48.084578 17346 webserver.cc:460] Webserver started at http://127.0.0.1:39167/ using document root <none> and password file <none>
I20260810 14:49:48.085264 17346 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:48.085409 17346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:48.085719 17346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:48.088740 17346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "7e52fe68f8d345aa9cb2a0c9cadc21ae"
format_stamp: "Formatted at 2026-08-10 14:49:48 on dist-test-slave-vkjp"
server_key: "7c9f9fdd504104806fda272b726a3315"
server_key_iv: "43d55bdfb90239fb51aa9dceb27ef4da"
server_key_version: "encryptionkey@0"
I20260810 14:49:48.089509 17346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "7e52fe68f8d345aa9cb2a0c9cadc21ae"
format_stamp: "Formatted at 2026-08-10 14:49:48 on dist-test-slave-vkjp"
server_key: "7c9f9fdd504104806fda272b726a3315"
server_key_iv: "43d55bdfb90239fb51aa9dceb27ef4da"
server_key_version: "encryptionkey@0"
I20260810 14:49:48.094799 17346 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 14:49:48.098239 17364 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:48.099627 17346 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:49:48.099812 17346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "7e52fe68f8d345aa9cb2a0c9cadc21ae"
format_stamp: "Formatted at 2026-08-10 14:49:48 on dist-test-slave-vkjp"
server_key: "7c9f9fdd504104806fda272b726a3315"
server_key_iv: "43d55bdfb90239fb51aa9dceb27ef4da"
server_key_version: "encryptionkey@0"
I20260810 14:49:48.100024 17346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:48.105671 17346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:48.109481 17346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:48.109781 17346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:48.128530 17346 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37163
I20260810 14:49:48.128575 17424 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37163 every 8 connection(s)
I20260810 14:49:48.129959 17346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:49:48.133716 17425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:48.139569 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 17346
I20260810 14:49:48.139767 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:49:48.140381 25438 external_mini_cluster.cc:1428] Setting key 56b5b5f77a6b2eaa45f00d015840193f
I20260810 14:49:48.145423 17425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae: Bootstrap starting.
I20260810 14:49:48.148336 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:48.142275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51639 (local address 127.0.0.1:37163)
0810 14:49:48.143162 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:48.143174 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:48.143249 (+ 75us) server_negotiation.cc:407] Connection header received
0810 14:49:48.144984 (+ 1735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:48.145008 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:48.145344 (+ 336us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:48.145709 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:48.146800 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:48.146806 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:48.146818 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:48.146832 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:48.146888 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:48.146908 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:48.147073 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:48.147239 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 14:49:48.147436 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":609,"thread_start_us":374,"threads_started":1}
I20260810 14:49:48.149360 17425 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:48.150545 17425 log.cc:824] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:48.152058 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:49:48.153825 17425 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae: No bootstrap required, opened a new log
I20260810 14:49:48.159806 17429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:48.157163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37507 (local address 127.0.0.1:37163)
0810 14:49:48.157381 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:48.157391 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:48.157823 (+ 432us) server_negotiation.cc:407] Connection header received
0810 14:49:48.157987 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:48.157991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:48.158037 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:48.158122 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:48.159067 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:48.159071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:48.159074 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:48.159142 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:48.159187 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:48.159203 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:48.159293 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:48.159428 (+ 135us) server_negotiation.cc:299] Negotiation successful
0810 14:49:48.159479 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:49:48.163751 17425 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e52fe68f8d345aa9cb2a0c9cadc21ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } }
I20260810 14:49:48.164141 17425 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:48.164258 17425 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e52fe68f8d345aa9cb2a0c9cadc21ae, State: Initialized, Role: FOLLOWER
I20260810 14:49:48.164882 17425 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e52fe68f8d345aa9cb2a0c9cadc21ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } }
I20260810 14:49:48.165132 17425 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:49:48.165284 17425 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:49:48.165453 17425 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:48.168051 17425 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e52fe68f8d345aa9cb2a0c9cadc21ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } }
I20260810 14:49:48.168497 17425 leader_election.cc:302] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e52fe68f8d345aa9cb2a0c9cadc21ae; no voters:
I20260810 14:49:48.169044 17425 leader_election.cc:288] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:49:48.169173 17436 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:48.169934 17436 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [term 1 LEADER]: Becoming Leader. State: Replica: 7e52fe68f8d345aa9cb2a0c9cadc21ae, State: Running, Role: LEADER
I20260810 14:49:48.170542 17436 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e52fe68f8d345aa9cb2a0c9cadc21ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } }
I20260810 14:49:48.171348 17425 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:49:48.173475 17437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e52fe68f8d345aa9cb2a0c9cadc21ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e52fe68f8d345aa9cb2a0c9cadc21ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } } }
I20260810 14:49:48.173869 17438 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e52fe68f8d345aa9cb2a0c9cadc21ae. Latest consensus state: current_term: 1 leader_uuid: "7e52fe68f8d345aa9cb2a0c9cadc21ae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e52fe68f8d345aa9cb2a0c9cadc21ae" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37163 } } }
I20260810 14:49:48.174175 17437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:48.174252 17438 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:48.179076 17441 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:49:48.184643 17441 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:49:48.193424 17441 catalog_manager.cc:1347] Generated new cluster ID: 648d88ceabd041f398bb78c493816c0c
I20260810 14:49:48.193544 17441 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:49:48.235335 17441 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:49:48.235781 17441 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:49:48.243994 17441 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e52fe68f8d345aa9cb2a0c9cadc21ae: Generated new TSK 0
I20260810 14:49:48.244822 17441 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:49:48.261056 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 17346
W20260810 14:49:48.262529 17420 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:51639
W20260810 14:49:48.263517 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 17346: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:49:48.263660 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 17346
2026-08-10T14:49:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2068 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:49:48 dist-test-slave-vkjp krb5kdc[17457](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:49:48 dist-test-slave-vkjp krb5kdc[17457](info): set up 2 sockets
Aug 10 14:49:48 dist-test-slave-vkjp krb5kdc[17457](info): commencing operation
krb5kdc: starting...
W20260810 14:49:49.468761 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:49:49 dist-test-slave-vkjp krb5kdc[17457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:49:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:49:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:49:49.772780 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41475
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40585
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41475
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:50.117846 17471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:50.118145 17471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:50.118319 17471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:50.125537 17471 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:49:50.125641 17471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:50.125687 17471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:50.125793 17471 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:49:50.125854 17471 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:49:50.135686 17471 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40585
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41475
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41475
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:50.137087 17471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:50.138414 17471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:50.150306 17478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:50.150233 17477 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:50.150729 17480 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:50.151813 17471 server_base.cc:1034] running on GCE node
I20260810 14:49:50.152566 17471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:50.153932 17471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:50.155166 17471 hybrid_clock.cc:648] HybridClock initialized: now 1786373390155104 us; error 54 us; skew 500 ppm
I20260810 14:49:50.155519 17471 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:50 dist-test-slave-vkjp krb5kdc[17457](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373390, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:50.162111 17471 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:49:50.163617 17471 webserver.cc:460] Webserver started at http://127.0.0.1:41669/ using document root <none> and password file <none>
I20260810 14:49:50.164301 17471 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:50.164449 17471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:50.164765 17471 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:50.167799 17471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "3d598d6225824b3495e87615c2fa1a44"
format_stamp: "Formatted at 2026-08-10 14:49:50 on dist-test-slave-vkjp"
server_key: "8830a44cb8d359a7daacea39dde00aaf"
server_key_iv: "47c5c483907abda954411ced121fa941"
server_key_version: "encryptionkey@0"
I20260810 14:49:50.168567 17471 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "3d598d6225824b3495e87615c2fa1a44"
format_stamp: "Formatted at 2026-08-10 14:49:50 on dist-test-slave-vkjp"
server_key: "8830a44cb8d359a7daacea39dde00aaf"
server_key_iv: "47c5c483907abda954411ced121fa941"
server_key_version: "encryptionkey@0"
I20260810 14:49:50.173660 17471 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:49:50.177122 17491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:50.178419 17471 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:49:50.178609 17471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "3d598d6225824b3495e87615c2fa1a44"
format_stamp: "Formatted at 2026-08-10 14:49:50 on dist-test-slave-vkjp"
server_key: "8830a44cb8d359a7daacea39dde00aaf"
server_key_iv: "47c5c483907abda954411ced121fa941"
server_key_version: "encryptionkey@0"
I20260810 14:49:50.178812 17471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:50.199321 17471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:50.203717 17471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:50.204027 17471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:50.222761 17471 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41475
I20260810 14:49:50.222795 17546 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41475 every 8 connection(s)
I20260810 14:49:50.224175 17471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:49:50.227862 17547 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:50.232467 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 17471
I20260810 14:49:50.232657 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:49:50.233234 25438 external_mini_cluster.cc:1428] Setting key a21a8e6692f9738df086c013f7ca2085
I20260810 14:49:50.239656 17547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44: Bootstrap starting.
I20260810 14:49:50.243847 17547 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:50.245137 17547 log.cc:824] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:50.248667 17547 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44: No bootstrap required, opened a new log
Aug 10 14:49:50 dist-test-slave-vkjp krb5kdc[17457](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373389, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 14:49:50.257697 17547 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d598d6225824b3495e87615c2fa1a44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41475 } }
I20260810 14:49:50.258138 17547 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:50.258256 17547 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d598d6225824b3495e87615c2fa1a44, State: Initialized, Role: FOLLOWER
I20260810 14:49:50.258984 17547 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d598d6225824b3495e87615c2fa1a44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41475 } }
I20260810 14:49:50.259295 17547 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:49:50.259457 17547 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:49:50.259642 17547 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:50.260334 17550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:50.235048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52597 (local address 127.0.0.1:41475)
0810 14:49:50.235865 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:50.235877 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:50.235916 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:49:50.238055 (+ 2139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:50.238077 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:50.238437 (+ 360us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:50.238833 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:50.240868 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:50.242985 (+ 2117us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:50.244973 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:50.246137 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:50.247148 (+ 1011us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:50.252515 (+ 5367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:50.252523 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:50.252537 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:50.252585 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:50.255896 (+ 3311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:50.256744 (+ 848us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:50.256753 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:50.256759 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:50.256915 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:50.257347 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:50.257350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:50.257352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:50.257855 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:50.258110 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:50.258651 (+ 541us) server_negotiation.cc:299] Negotiation successful
0810 14:49:50.259112 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":365,"threads_started":1}
I20260810 14:49:50.262152 17547 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d598d6225824b3495e87615c2fa1a44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41475 } }
I20260810 14:49:50.262702 17547 leader_election.cc:302] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3d598d6225824b3495e87615c2fa1a44; no voters:
I20260810 14:49:50.263445 17547 leader_election.cc:288] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:49:50.263553 17552 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:50.264161 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:49:50.266108 17547 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:49:50.270179 17552 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [term 1 LEADER]: Becoming Leader. State: Replica: 3d598d6225824b3495e87615c2fa1a44, State: Running, Role: LEADER
I20260810 14:49:50.270893 17552 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d598d6225824b3495e87615c2fa1a44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41475 } }
I20260810 14:49:50.276741 17553 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3d598d6225824b3495e87615c2fa1a44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d598d6225824b3495e87615c2fa1a44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41475 } } }
I20260810 14:49:50.277262 17553 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:50.277439 17554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3d598d6225824b3495e87615c2fa1a44. Latest consensus state: current_term: 1 leader_uuid: "3d598d6225824b3495e87615c2fa1a44" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3d598d6225824b3495e87615c2fa1a44" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41475 } } }
I20260810 14:49:50.277784 17554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:50.282727 17567 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:49:50.286365 17550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:50.268678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48039 (local address 127.0.0.1:41475)
0810 14:49:50.268862 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:50.268870 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:50.269361 (+ 491us) server_negotiation.cc:407] Connection header received
0810 14:49:50.269587 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:50.269592 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:50.269663 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:50.269762 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:50.271229 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:50.272042 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:50.273683 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:50.274178 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:50.274309 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:50.278848 (+ 4539us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:50.278858 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:50.278867 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:50.278921 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:50.282945 (+ 4024us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:50.283833 (+ 888us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:50.283838 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:50.283840 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:50.283957 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:50.284375 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:50.284378 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:50.284380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:50.284749 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:50.285049 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:50.285704 (+ 655us) server_negotiation.cc:299] Negotiation successful
0810 14:49:50.285970 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 14:49:50.288738 17567 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:49:50.297758 17567 catalog_manager.cc:1347] Generated new cluster ID: f4430ef15f974da2b0ba73aa9148b447
I20260810 14:49:50.297900 17567 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:49:50.313966 17567 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:49:50.315359 17567 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:49:50.326143 17567 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3d598d6225824b3495e87615c2fa1a44: Generated new TSK 0
I20260810 14:49:50.327019 17567 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:49:50.336853 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 17471
I20260810 14:49:50.338112 17542 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:50.429427 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 17471
2026-08-10T14:49:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2165 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:49:50 dist-test-slave-vkjp krb5kdc[17585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:49:50 dist-test-slave-vkjp krb5kdc[17585](info): set up 2 sockets
Aug 10 14:49:50 dist-test-slave-vkjp krb5kdc[17585](info): commencing operation
krb5kdc: starting...
W20260810 14:49:51.677417 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.209s user 0.002s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:49:51 dist-test-slave-vkjp krb5kdc[17585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373391, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:49:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:49:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:49:51.992750 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35193
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:52.337440 17598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:52.337751 17598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:52.337942 17598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:52.345377 17598 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:49:52.345500 17598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:52.345605 17598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:52.345716 17598 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:49:52.345793 17598 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:49:52.356539 17598 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:52.357899 17598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:52.359310 17598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:52.371799 17604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:52.371856 17605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:52.372025 17607 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:52.372771 17598 server_base.cc:1034] running on GCE node
I20260810 14:49:52.373519 17598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:52.374871 17598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:52.376088 17598 hybrid_clock.cc:648] HybridClock initialized: now 1786373392376036 us; error 49 us; skew 500 ppm
I20260810 14:49:52.376437 17598 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:52 dist-test-slave-vkjp krb5kdc[17585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373392, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:52.379716 17598 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:49:52.381127 17598 webserver.cc:460] Webserver started at http://127.0.0.1:34413/ using document root <none> and password file <none>
I20260810 14:49:52.381767 17598 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:52.381906 17598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:52.382211 17598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:52.385263 17598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "3fec506389ff4cfda01c283c4612ef47"
format_stamp: "Formatted at 2026-08-10 14:49:52 on dist-test-slave-vkjp"
server_key: "ea6c031dcbab653e181663acd6b51d84"
server_key_iv: "9ff9474a04ab23a85130ccc16b29eb75"
server_key_version: "encryptionkey@0"
I20260810 14:49:52.385964 17598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "3fec506389ff4cfda01c283c4612ef47"
format_stamp: "Formatted at 2026-08-10 14:49:52 on dist-test-slave-vkjp"
server_key: "ea6c031dcbab653e181663acd6b51d84"
server_key_iv: "9ff9474a04ab23a85130ccc16b29eb75"
server_key_version: "encryptionkey@0"
I20260810 14:49:52.391012 17598 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 14:49:52.394425 17614 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:52.395712 17598 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 14:49:52.395845 17598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "3fec506389ff4cfda01c283c4612ef47"
format_stamp: "Formatted at 2026-08-10 14:49:52 on dist-test-slave-vkjp"
server_key: "ea6c031dcbab653e181663acd6b51d84"
server_key_iv: "9ff9474a04ab23a85130ccc16b29eb75"
server_key_version: "encryptionkey@0"
I20260810 14:49:52.396006 17598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:52.412024 17598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:52.416509 17598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:52.416759 17598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:52.435678 17598 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35193
I20260810 14:49:52.435724 17669 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35193 every 8 connection(s)
I20260810 14:49:52.437050 17598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:49:52.440743 17670 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:52.443306 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 17598
I20260810 14:49:52.443495 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:49:52.443928 25438 external_mini_cluster.cc:1428] Setting key c0462937e1814f14323c4986fc9f37ae
I20260810 14:49:52.452502 17670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47: Bootstrap starting.
I20260810 14:49:52.456163 17670 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:52.457161 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:52.445771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47981 (local address 127.0.0.1:35193)
0810 14:49:52.446578 (+ 807us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:52.446589 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:52.446626 (+ 37us) server_negotiation.cc:407] Connection header received
0810 14:49:52.448284 (+ 1658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:52.448310 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:52.448680 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:52.449002 (+ 322us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:52.449646 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:52.450951 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:52.452854 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:52.453503 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:52.453627 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:52.454863 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:52.454871 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:52.454883 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:52.455473 (+ 590us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:52.455546 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:52.455575 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:52.455660 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:52.455891 (+ 231us) server_negotiation.cc:299] Negotiation successful
0810 14:49:52.456159 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":384,"threads_started":1}
I20260810 14:49:52.457297 17670 log.cc:824] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:52.460479 17670 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47: No bootstrap required, opened a new log
I20260810 14:49:52.460928 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:49:52.469645 17670 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fec506389ff4cfda01c283c4612ef47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35193 } }
I20260810 14:49:52.470054 17670 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:52.470191 17670 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fec506389ff4cfda01c283c4612ef47, State: Initialized, Role: FOLLOWER
I20260810 14:49:52.470860 17670 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fec506389ff4cfda01c283c4612ef47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35193 } }
I20260810 14:49:52.471187 17670 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:49:52.471357 17670 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:49:52.471558 17670 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:52.471971 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:52.465182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44227 (local address 127.0.0.1:35193)
0810 14:49:52.465396 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:52.465405 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:52.465862 (+ 457us) server_negotiation.cc:407] Connection header received
0810 14:49:52.466058 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:52.466062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:52.466123 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:52.466199 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:52.466791 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:52.467621 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:52.469458 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:52.469872 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:52.469962 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:52.471196 (+ 1234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:52.471200 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:52.471203 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:52.471220 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:52.471255 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:52.471291 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:52.471377 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:52.471568 (+ 191us) server_negotiation.cc:299] Negotiation successful
0810 14:49:52.471635 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:49:52.474016 17670 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fec506389ff4cfda01c283c4612ef47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35193 } }
I20260810 14:49:52.474517 17670 leader_election.cc:302] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fec506389ff4cfda01c283c4612ef47; no voters:
I20260810 14:49:52.475147 17670 leader_election.cc:288] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:49:52.475296 17680 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:52.476037 17680 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [term 1 LEADER]: Becoming Leader. State: Replica: 3fec506389ff4cfda01c283c4612ef47, State: Running, Role: LEADER
I20260810 14:49:52.476599 17680 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fec506389ff4cfda01c283c4612ef47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35193 } }
I20260810 14:49:52.477253 17670 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:49:52.480544 17681 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fec506389ff4cfda01c283c4612ef47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fec506389ff4cfda01c283c4612ef47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35193 } } }
I20260810 14:49:52.480949 17681 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:52.481452 17682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fec506389ff4cfda01c283c4612ef47. Latest consensus state: current_term: 1 leader_uuid: "3fec506389ff4cfda01c283c4612ef47" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fec506389ff4cfda01c283c4612ef47" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35193 } } }
I20260810 14:49:52.481765 17682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:52.482123 17689 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:49:52.489835 17689 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:49:52.501847 17689 catalog_manager.cc:1347] Generated new cluster ID: 7e5860f64afb41e3acabfb9bfd0f715c
I20260810 14:49:52.501979 17689 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:49:52.512446 17689 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:49:52.514321 17689 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:49:52.522826 17689 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fec506389ff4cfda01c283c4612ef47: Generated new TSK 0
I20260810 14:49:52.523900 17689 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:49:52.544751 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 17598
W20260810 14:49:52.546607 17665 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47981
W20260810 14:49:52.547335 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 17598: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:49:52.547467 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 17598
2026-08-10T14:49:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2120 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:49:52 dist-test-slave-vkjp krb5kdc[17703](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:49:52 dist-test-slave-vkjp krb5kdc[17703](info): set up 2 sockets
Aug 10 14:49:52 dist-test-slave-vkjp krb5kdc[17703](info): commencing operation
krb5kdc: starting...
W20260810 14:49:53.747856 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.003s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:49:53 dist-test-slave-vkjp krb5kdc[17703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373393, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:49:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:49:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:49:54.063210 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42445
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:54.403846 17721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:54.404232 17721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:54.404417 17721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:54.411526 17721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:49:54.411674 17721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:54.411758 17721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:54.411864 17721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:49:54.411943 17721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:49:54.421715 17721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42445
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:54.423102 17721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:54.424427 17721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:54.436216 17728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:54.436245 17729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:54.436264 17731 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:54.437541 17721 server_base.cc:1034] running on GCE node
I20260810 14:49:54.438246 17721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:54.439651 17721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:54.440882 17721 hybrid_clock.cc:648] HybridClock initialized: now 1786373394440846 us; error 42 us; skew 500 ppm
I20260810 14:49:54.441239 17721 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:54 dist-test-slave-vkjp krb5kdc[17703](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373394, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:54.444386 17721 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:49:54.445883 17721 webserver.cc:460] Webserver started at http://127.0.0.1:34545/ using document root <none> and password file <none>
I20260810 14:49:54.446542 17721 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:54.446696 17721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:54.447010 17721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:54.449965 17721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "41a632360887468e90a1aa5fa96db737"
format_stamp: "Formatted at 2026-08-10 14:49:54 on dist-test-slave-vkjp"
server_key: "e7c960fcc0de07afa862f4092eb6611c"
server_key_iv: "b8171f8b49e3494261d626ef4c3ee81c"
server_key_version: "encryptionkey@0"
I20260810 14:49:54.450740 17721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "41a632360887468e90a1aa5fa96db737"
format_stamp: "Formatted at 2026-08-10 14:49:54 on dist-test-slave-vkjp"
server_key: "e7c960fcc0de07afa862f4092eb6611c"
server_key_iv: "b8171f8b49e3494261d626ef4c3ee81c"
server_key_version: "encryptionkey@0"
I20260810 14:49:54.455925 17721 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.001s
I20260810 14:49:54.459365 17741 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:54.460610 17721 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260810 14:49:54.460788 17721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "41a632360887468e90a1aa5fa96db737"
format_stamp: "Formatted at 2026-08-10 14:49:54 on dist-test-slave-vkjp"
server_key: "e7c960fcc0de07afa862f4092eb6611c"
server_key_iv: "b8171f8b49e3494261d626ef4c3ee81c"
server_key_version: "encryptionkey@0"
I20260810 14:49:54.461001 17721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:54.467069 17721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:54.470993 17721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:54.471315 17721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:54.490092 17721 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42445
I20260810 14:49:54.490136 17797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42445 every 8 connection(s)
I20260810 14:49:54.491578 17721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:49:54.493466 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 17721
I20260810 14:49:54.493646 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:49:54.494105 25438 external_mini_cluster.cc:1428] Setting key cde34ad6eaf42d858248de23049c4b36
I20260810 14:49:54.495729 17798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:54.503589 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:54.495901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48297 (local address 127.0.0.1:42445)
0810 14:49:54.496710 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:54.496729 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:54.496782 (+ 53us) server_negotiation.cc:407] Connection header received
0810 14:49:54.498922 (+ 2140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:54.498949 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:54.499424 (+ 475us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:54.499876 (+ 452us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:54.501609 (+ 1733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:54.501618 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:54.501636 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:54.501661 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:54.501774 (+ 113us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:54.501805 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:54.502045 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:54.502189 (+ 144us) server_negotiation.cc:299] Negotiation successful
0810 14:49:54.502388 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":312,"threads_started":1}
I20260810 14:49:54.507478 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:49:54.510653 17798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737: Bootstrap starting.
I20260810 14:49:54.514993 17798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:54.515772 17801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:54.513095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37413 (local address 127.0.0.1:42445)
0810 14:49:54.513310 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:54.513316 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:54.513610 (+ 294us) server_negotiation.cc:407] Connection header received
0810 14:49:54.513806 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:54.513810 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:54.513860 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:54.513941 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:54.514985 (+ 1044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:54.514991 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:54.514994 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:54.515011 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:54.515053 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:54.515134 (+ 81us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:54.515288 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:54.515443 (+ 155us) server_negotiation.cc:299] Negotiation successful
0810 14:49:54.515514 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260810 14:49:54.516323 17798 log.cc:824] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:54.519382 17798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737: No bootstrap required, opened a new log
I20260810 14:49:54.527205 17798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a632360887468e90a1aa5fa96db737" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42445 } }
I20260810 14:49:54.527710 17798 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:54.527860 17798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41a632360887468e90a1aa5fa96db737, State: Initialized, Role: FOLLOWER
I20260810 14:49:54.528445 17798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a632360887468e90a1aa5fa96db737" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42445 } }
I20260810 14:49:54.528662 17798 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:49:54.528765 17798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:49:54.528887 17798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:54.531059 17798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a632360887468e90a1aa5fa96db737" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42445 } }
I20260810 14:49:54.531598 17798 leader_election.cc:302] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41a632360887468e90a1aa5fa96db737; no voters:
I20260810 14:49:54.532102 17798 leader_election.cc:288] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:49:54.532258 17809 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:54.532940 17809 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [term 1 LEADER]: Becoming Leader. State: Replica: 41a632360887468e90a1aa5fa96db737, State: Running, Role: LEADER
I20260810 14:49:54.533491 17809 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a632360887468e90a1aa5fa96db737" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42445 } }
I20260810 14:49:54.534273 17798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:49:54.536535 17810 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41a632360887468e90a1aa5fa96db737" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a632360887468e90a1aa5fa96db737" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42445 } } }
I20260810 14:49:54.536573 17811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41a632360887468e90a1aa5fa96db737. Latest consensus state: current_term: 1 leader_uuid: "41a632360887468e90a1aa5fa96db737" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41a632360887468e90a1aa5fa96db737" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42445 } } }
I20260810 14:49:54.536981 17811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:54.536993 17810 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:54.540377 17817 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:49:54.545437 17817 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:49:54.554280 17817 catalog_manager.cc:1347] Generated new cluster ID: 8f3f0b5f0a53434dad00043b69874c43
I20260810 14:49:54.554409 17817 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:49:54.587270 17817 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:49:54.587821 17817 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:49:54.596706 17817 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41a632360887468e90a1aa5fa96db737: Generated new TSK 0
I20260810 14:49:54.597733 17817 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:49:54.607591 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 17721
W20260810 14:49:54.609242 17792 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:48297
W20260810 14:49:54.609994 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 17721: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:49:54.610158 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 17721
2026-08-10T14:49:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2063 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:49:54 dist-test-slave-vkjp krb5kdc[17833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:49:54 dist-test-slave-vkjp krb5kdc[17833](info): set up 2 sockets
Aug 10 14:49:54 dist-test-slave-vkjp krb5kdc[17833](info): commencing operation
krb5kdc: starting...
W20260810 14:49:55.830533 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.177s user 0.000s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:49:56 dist-test-slave-vkjp krb5kdc[17833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:49:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:49:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:49:56.138470 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37419
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:56.481787 17847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:56.482128 17847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:56.482300 17847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:56.489537 17847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:49:56.489696 17847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:56.489804 17847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:56.489912 17847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:49:56.489987 17847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:49:56.500473 17847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37419
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:56.501897 17847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:56.503345 17847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:56.516173 17853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:56.516284 17855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:56.516660 17852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:56.517390 17847 server_base.cc:1034] running on GCE node
I20260810 14:49:56.518244 17847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:56.519841 17847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:56.521081 17847 hybrid_clock.cc:648] HybridClock initialized: now 1786373396521028 us; error 78 us; skew 500 ppm
I20260810 14:49:56.521445 17847 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:56 dist-test-slave-vkjp krb5kdc[17833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373396, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:56.530203 17847 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:49:56.531870 17847 webserver.cc:460] Webserver started at http://127.0.0.1:42497/ using document root <none> and password file <none>
I20260810 14:49:56.532593 17847 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:56.532730 17847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:56.533071 17847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:56.536239 17847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "e75a40cc67664c40add12bdae6c4c51e"
format_stamp: "Formatted at 2026-08-10 14:49:56 on dist-test-slave-vkjp"
server_key: "bdc5bf930ae3991d57f8f4d77eb7c46e"
server_key_iv: "c76a10b051982b2edf3307b81bdac4fb"
server_key_version: "encryptionkey@0"
I20260810 14:49:56.537037 17847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "e75a40cc67664c40add12bdae6c4c51e"
format_stamp: "Formatted at 2026-08-10 14:49:56 on dist-test-slave-vkjp"
server_key: "bdc5bf930ae3991d57f8f4d77eb7c46e"
server_key_iv: "c76a10b051982b2edf3307b81bdac4fb"
server_key_version: "encryptionkey@0"
I20260810 14:49:56.542305 17847 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260810 14:49:56.545787 17862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:56.547174 17847 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260810 14:49:56.547362 17847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "e75a40cc67664c40add12bdae6c4c51e"
format_stamp: "Formatted at 2026-08-10 14:49:56 on dist-test-slave-vkjp"
server_key: "bdc5bf930ae3991d57f8f4d77eb7c46e"
server_key_iv: "c76a10b051982b2edf3307b81bdac4fb"
server_key_version: "encryptionkey@0"
I20260810 14:49:56.547550 17847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:56.563895 17847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:56.567764 17847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:56.568054 17847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:56.586613 17847 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37419
I20260810 14:49:56.586701 17924 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37419 every 8 connection(s)
I20260810 14:49:56.588037 17847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:49:56.588853 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 17847
I20260810 14:49:56.589066 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:49:56.589635 25438 external_mini_cluster.cc:1428] Setting key 97ef95b920c9b3377dd2defd549dee44
I20260810 14:49:56.592648 17925 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 10 14:49:56 dist-test-slave-vkjp krb5kdc[17833](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373396, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 14:49:56.605073 17925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e: Bootstrap starting.
I20260810 14:49:56.609383 17925 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:56.610671 17925 log.cc:824] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:56.614027 17925 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e: No bootstrap required, opened a new log
I20260810 14:49:56.614285 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:56.591471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:47593 (local address 127.0.0.1:37419)
0810 14:49:56.592279 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:56.592294 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:56.592345 (+ 51us) server_negotiation.cc:407] Connection header received
0810 14:49:56.594638 (+ 2293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:56.594666 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:56.595175 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:56.595609 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:56.596856 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:56.598904 (+ 2048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:56.600278 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:56.601310 (+ 1032us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:56.601491 (+ 181us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:56.605636 (+ 4145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:56.605647 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:56.605665 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:56.605708 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:56.609711 (+ 4003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:56.610522 (+ 811us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:56.610530 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:56.610537 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:56.610753 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:56.611337 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:56.611344 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:56.611347 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:56.611911 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:56.612271 (+ 360us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:56.612657 (+ 386us) server_negotiation.cc:299] Negotiation successful
0810 14:49:56.612993 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":330,"threads_started":1}
I20260810 14:49:56.618464 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:49:56.624616 17925 raft_consensus.cc:348] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e75a40cc67664c40add12bdae6c4c51e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37419 } }
I20260810 14:49:56.625016 17925 raft_consensus.cc:374] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:56.625131 17925 raft_consensus.cc:729] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e75a40cc67664c40add12bdae6c4c51e, State: Initialized, Role: FOLLOWER
I20260810 14:49:56.625761 17925 consensus_queue.cc:260] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e75a40cc67664c40add12bdae6c4c51e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37419 } }
I20260810 14:49:56.626012 17925 raft_consensus.cc:388] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:49:56.626164 17925 raft_consensus.cc:482] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:49:56.626362 17925 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:56.628681 17925 raft_consensus.cc:504] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e75a40cc67664c40add12bdae6c4c51e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37419 } }
I20260810 14:49:56.629184 17925 leader_election.cc:302] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e75a40cc67664c40add12bdae6c4c51e; no voters:
I20260810 14:49:56.629859 17925 leader_election.cc:288] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:49:56.629966 17935 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:56.631839 17935 raft_consensus.cc:686] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [term 1 LEADER]: Becoming Leader. State: Replica: e75a40cc67664c40add12bdae6c4c51e, State: Running, Role: LEADER
I20260810 14:49:56.632223 17925 sys_catalog.cc:564] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:49:56.632510 17935 consensus_queue.cc:237] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e75a40cc67664c40add12bdae6c4c51e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37419 } }
I20260810 14:49:56.638470 17936 sys_catalog.cc:455] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e75a40cc67664c40add12bdae6c4c51e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e75a40cc67664c40add12bdae6c4c51e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37419 } } }
I20260810 14:49:56.638887 17936 sys_catalog.cc:458] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:56.639360 17937 sys_catalog.cc:455] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e75a40cc67664c40add12bdae6c4c51e. Latest consensus state: current_term: 1 leader_uuid: "e75a40cc67664c40add12bdae6c4c51e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e75a40cc67664c40add12bdae6c4c51e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37419 } } }
I20260810 14:49:56.639657 17937 sys_catalog.cc:458] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:56.641485 17945 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:49:56.643852 17928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:56.623853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:49883 (local address 127.0.0.1:37419)
0810 14:49:56.624109 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:56.624118 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:56.625572 (+ 1454us) server_negotiation.cc:407] Connection header received
0810 14:49:56.625782 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:56.625786 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:56.625849 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:56.625953 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:56.627544 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:56.628338 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:56.629701 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:56.630489 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:56.631269 (+ 780us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:56.634098 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:56.634110 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:56.634118 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:49:56.634164 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:56.639582 (+ 5418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:56.640704 (+ 1122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:56.640709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:56.640712 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:56.640972 (+ 260us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:49:56.641639 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:49:56.641646 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:49:56.641650 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:49:56.642141 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:56.643308 (+ 1167us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:56.643467 (+ 159us) server_negotiation.cc:299] Negotiation successful
0810 14:49:56.643606 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 14:49:56.647185 17945 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:49:56.656083 17945 catalog_manager.cc:1347] Generated new cluster ID: 696349d277de4b78aadbcab4cc3f83e3
I20260810 14:49:56.656203 17945 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:49:56.678295 17945 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:49:56.680171 17945 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:49:56.690264 17945 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e75a40cc67664c40add12bdae6c4c51e: Generated new TSK 0
I20260810 14:49:56.691169 17945 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:49:56.698185 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 17847
I20260810 14:49:56.699337 17919 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:49:56.789294 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 17847
W20260810 14:49:56.811695 17831 connection.cc:535] client connection to 127.0.0.1:37419 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37419): Connection reset by peer (error 104)
2026-08-10T14:49:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2177 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:49:56 dist-test-slave-vkjp krb5kdc[17959](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:49:56 dist-test-slave-vkjp krb5kdc[17959](info): set up 2 sockets
Aug 10 14:49:56 dist-test-slave-vkjp krb5kdc[17959](info): commencing operation
krb5kdc: starting...
W20260810 14:49:58.018522 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:49:58 dist-test-slave-vkjp krb5kdc[17959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373398, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:49:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:49:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:49:58.330118 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37033
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36523
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:49:58.680972 17972 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:49:58.681332 17972 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:49:58.681504 17972 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:49:58.688794 17972 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:49:58.688956 17972 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:49:58.689059 17972 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:49:58.689170 17972 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:49:58.689247 17972 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:49:58.699232 17972 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36523
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:49:58.700721 17972 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:49:58.702049 17972 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:49:58.714496 17978 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:58.714526 17980 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:49:58.714740 17977 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:49:58.715276 17972 server_base.cc:1034] running on GCE node
I20260810 14:49:58.716065 17972 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:49:58.717475 17972 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:49:58.718705 17972 hybrid_clock.cc:648] HybridClock initialized: now 1786373398718662 us; error 54 us; skew 500 ppm
I20260810 14:49:58.719053 17972 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:49:58 dist-test-slave-vkjp krb5kdc[17959](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373398, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:49:58.722461 17972 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:49:58.724032 17972 webserver.cc:460] Webserver started at http://127.0.0.1:40803/ using document root <none> and password file <none>
I20260810 14:49:58.724685 17972 fs_manager.cc:359] Metadata directory not provided
I20260810 14:49:58.724793 17972 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:49:58.725054 17972 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:49:58.728150 17972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "54bf72e6594d442da30549a8bbec6190"
format_stamp: "Formatted at 2026-08-10 14:49:58 on dist-test-slave-vkjp"
server_key: "44b044e61e7731ec5a3b39d15ddb6d4e"
server_key_iv: "8d9cce118833d8cec64e2e95b523b2e4"
server_key_version: "encryptionkey@0"
I20260810 14:49:58.728888 17972 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "54bf72e6594d442da30549a8bbec6190"
format_stamp: "Formatted at 2026-08-10 14:49:58 on dist-test-slave-vkjp"
server_key: "44b044e61e7731ec5a3b39d15ddb6d4e"
server_key_iv: "8d9cce118833d8cec64e2e95b523b2e4"
server_key_version: "encryptionkey@0"
I20260810 14:49:58.734216 17972 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:49:58.737659 17987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:49:58.738991 17972 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:49:58.739207 17972 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "54bf72e6594d442da30549a8bbec6190"
format_stamp: "Formatted at 2026-08-10 14:49:58 on dist-test-slave-vkjp"
server_key: "44b044e61e7731ec5a3b39d15ddb6d4e"
server_key_iv: "8d9cce118833d8cec64e2e95b523b2e4"
server_key_version: "encryptionkey@0"
I20260810 14:49:58.739444 17972 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:49:58.776566 17972 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:49:58.780857 17972 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:49:58.781208 17972 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:49:58.800226 17972 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37033
I20260810 14:49:58.800261 18044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37033 every 8 connection(s)
I20260810 14:49:58.801738 17972 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:49:58.802332 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 17972
I20260810 14:49:58.802561 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:49:58.803193 25438 external_mini_cluster.cc:1428] Setting key 6e9a6ecc345d1bc6701113fb77f14764
I20260810 14:49:58.806373 18045 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:49:58.816663 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:58.805103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:51713 (local address 127.0.0.1:37033)
0810 14:49:58.805875 (+ 772us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:58.805887 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:58.805928 (+ 41us) server_negotiation.cc:407] Connection header received
0810 14:49:58.807701 (+ 1773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:58.807725 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:58.808158 (+ 433us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:58.808544 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:58.809137 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:58.810477 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:58.811909 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:58.812627 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:58.812836 (+ 209us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:58.814272 (+ 1436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:58.814283 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:58.814298 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:58.814892 (+ 594us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:58.814990 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:58.815019 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:58.815248 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:58.815524 (+ 276us) server_negotiation.cc:299] Negotiation successful
0810 14:49:58.815720 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":338,"threads_started":1}
I20260810 14:49:58.817672 18045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190: Bootstrap starting.
I20260810 14:49:58.820845 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:49:58.821589 18045 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190: Neither blocks nor log segments found. Creating new log.
I20260810 14:49:58.822611 18045 log.cc:824] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190: Log is configured to *not* fsync() on all Append() calls
I20260810 14:49:58.825543 18045 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190: No bootstrap required, opened a new log
I20260810 14:49:58.832434 18048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:49:58.825382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:45913 (local address 127.0.0.1:37033)
0810 14:49:58.825583 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 14:49:58.825591 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:49:58.826144 (+ 553us) server_negotiation.cc:407] Connection header received
0810 14:49:58.826304 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:49:58.826310 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:49:58.826396 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:49:58.826524 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:49:58.827039 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:58.828305 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:58.829619 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:49:58.830315 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:49:58.830473 (+ 158us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:49:58.831587 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:49:58.831596 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:49:58.831602 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:49:58.831630 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:49:58.831693 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:49:58.831719 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:49:58.831853 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0810 14:49:58.832014 (+ 161us) server_negotiation.cc:299] Negotiation successful
0810 14:49:58.832071 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260810 14:49:58.835625 18045 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54bf72e6594d442da30549a8bbec6190" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37033 } }
I20260810 14:49:58.836014 18045 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:49:58.836138 18045 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54bf72e6594d442da30549a8bbec6190, State: Initialized, Role: FOLLOWER
I20260810 14:49:58.836777 18045 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54bf72e6594d442da30549a8bbec6190" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37033 } }
I20260810 14:49:58.837020 18045 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:49:58.837172 18045 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:49:58.837355 18045 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:49:58.839936 18045 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54bf72e6594d442da30549a8bbec6190" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37033 } }
I20260810 14:49:58.840564 18045 leader_election.cc:302] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54bf72e6594d442da30549a8bbec6190; no voters:
I20260810 14:49:58.841171 18045 leader_election.cc:288] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:49:58.841341 18056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:49:58.842072 18056 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [term 1 LEADER]: Becoming Leader. State: Replica: 54bf72e6594d442da30549a8bbec6190, State: Running, Role: LEADER
I20260810 14:49:58.842608 18056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54bf72e6594d442da30549a8bbec6190" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37033 } }
I20260810 14:49:58.843501 18045 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:49:58.845978 18058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54bf72e6594d442da30549a8bbec6190. Latest consensus state: current_term: 1 leader_uuid: "54bf72e6594d442da30549a8bbec6190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54bf72e6594d442da30549a8bbec6190" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37033 } } }
I20260810 14:49:58.846357 18058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:58.847069 18057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54bf72e6594d442da30549a8bbec6190" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54bf72e6594d442da30549a8bbec6190" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37033 } } }
I20260810 14:49:58.847458 18057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190 [sys.catalog]: This master's current role is: LEADER
I20260810 14:49:58.851864 18063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:49:58.857427 18063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:49:58.867280 18063 catalog_manager.cc:1347] Generated new cluster ID: 6933ff0a73e24d49bcdec6a06317e57b
I20260810 14:49:58.867455 18063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:49:58.894053 18063 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:49:58.895562 18063 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:49:58.904896 18063 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54bf72e6594d442da30549a8bbec6190: Generated new TSK 0
I20260810 14:49:58.905814 18063 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:49:58.921391 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 17972
W20260810 14:49:58.923309 18040 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.30:51713
W20260810 14:49:58.924103 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 17972: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:49:58.924258 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 17972
W20260810 14:49:58.946662 17957 connection.cc:535] client connection to 127.0.0.1:37033 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37033): Connection reset by peer (error 104)
2026-08-10T14:49:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2135 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:49:59 dist-test-slave-vkjp krb5kdc[18081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:49:59 dist-test-slave-vkjp krb5kdc[18081](info): set up 2 sockets
Aug 10 14:49:59 dist-test-slave-vkjp krb5kdc[18081](info): commencing operation
krb5kdc: starting...
W20260810 14:50:00.158380 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.191s user 0.003s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:50:00 dist-test-slave-vkjp krb5kdc[18081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373400, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:50:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:50:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:50:00.474709 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44225
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44225
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:50:00.844261 18096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:50:00.844563 18096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:50:00.844763 18096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:50:00.852190 18096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:50:00.852335 18096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:50:00.852430 18096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:50:00.852530 18096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:50:00.852602 18096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:50:00.862672 18096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44225
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44225
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:50:00.864140 18096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:50:00.865465 18096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:50:00.878223 18102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:00.878388 18103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:00.878520 18105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:50:00.880034 18096 server_base.cc:1034] running on GCE node
I20260810 14:50:00.880856 18096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:50:00.882298 18096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:50:00.883549 18096 hybrid_clock.cc:648] HybridClock initialized: now 1786373400883506 us; error 52 us; skew 500 ppm
I20260810 14:50:00.883946 18096 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:50:00 dist-test-slave-vkjp krb5kdc[18081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373400, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:50:00.887194 18096 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:50:00.888724 18096 webserver.cc:460] Webserver started at http://127.0.0.1:46741/ using document root <none> and password file <none>
I20260810 14:50:00.889400 18096 fs_manager.cc:359] Metadata directory not provided
I20260810 14:50:00.889552 18096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:50:00.889868 18096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:50:00.892944 18096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "d37446b885a44380ac241b882008e63a"
format_stamp: "Formatted at 2026-08-10 14:50:00 on dist-test-slave-vkjp"
server_key: "c1e0cd49e2ff30c24bf9163ad2d67348"
server_key_iv: "9a38734a79bef5ac58acb953b2c93741"
server_key_version: "encryptionkey@0"
I20260810 14:50:00.893747 18096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "d37446b885a44380ac241b882008e63a"
format_stamp: "Formatted at 2026-08-10 14:50:00 on dist-test-slave-vkjp"
server_key: "c1e0cd49e2ff30c24bf9163ad2d67348"
server_key_iv: "9a38734a79bef5ac58acb953b2c93741"
server_key_version: "encryptionkey@0"
I20260810 14:50:00.898900 18096 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 14:50:00.902340 18113 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:50:00.903689 18096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:50:00.903877 18096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "d37446b885a44380ac241b882008e63a"
format_stamp: "Formatted at 2026-08-10 14:50:00 on dist-test-slave-vkjp"
server_key: "c1e0cd49e2ff30c24bf9163ad2d67348"
server_key_iv: "9a38734a79bef5ac58acb953b2c93741"
server_key_version: "encryptionkey@0"
I20260810 14:50:00.904093 18096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:50:00.909976 18096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:50:00.914354 18096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:50:00.914674 18096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:50:00.934041 18096 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44225
I20260810 14:50:00.934082 18168 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44225 every 8 connection(s)
I20260810 14:50:00.935556 18096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:50:00.936671 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 18096
I20260810 14:50:00.936854 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:50:00.937412 25438 external_mini_cluster.cc:1428] Setting key ebcae763c8d51ae861d33c10f8fc5962
I20260810 14:50:00.940081 18169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:50:00.945232 18175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:00.939316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:53471 (local address 127.0.0.1:44225)
0810 14:50:00.940168 (+ 852us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:00.940186 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:00.940237 (+ 51us) server_negotiation.cc:407] Connection header received
0810 14:50:00.941910 (+ 1673us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:00.941932 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:00.942260 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:00.942591 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:00.943670 (+ 1079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:00.943676 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:00.943688 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:50:00.943701 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:00.943775 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:50:00.943799 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:00.943992 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:00.944099 (+ 107us) server_negotiation.cc:299] Negotiation successful
0810 14:50:00.944309 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":392,"threads_started":1}
I20260810 14:50:00.948987 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:50:00.953492 18169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a: Bootstrap starting.
I20260810 14:50:00.957242 18175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:00.954511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:49683 (local address 127.0.0.1:44225)
0810 14:50:00.954697 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:00.954703 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:00.955274 (+ 571us) server_negotiation.cc:407] Connection header received
0810 14:50:00.955472 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:00.955479 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:00.955550 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:00.955676 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:00.956590 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:00.956595 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:00.956600 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:50:00.956620 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:00.956667 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:50:00.956687 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:00.956803 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:00.956882 (+ 79us) server_negotiation.cc:299] Negotiation successful
0810 14:50:00.956952 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260810 14:50:00.957413 18169 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a: Neither blocks nor log segments found. Creating new log.
I20260810 14:50:00.958596 18169 log.cc:824] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a: Log is configured to *not* fsync() on all Append() calls
I20260810 14:50:00.961349 18169 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a: No bootstrap required, opened a new log
I20260810 14:50:00.968991 18169 raft_consensus.cc:348] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37446b885a44380ac241b882008e63a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44225 } }
I20260810 14:50:00.969420 18169 raft_consensus.cc:374] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:50:00.969493 18169 raft_consensus.cc:729] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d37446b885a44380ac241b882008e63a, State: Initialized, Role: FOLLOWER
I20260810 14:50:00.969998 18169 consensus_queue.cc:260] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37446b885a44380ac241b882008e63a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44225 } }
I20260810 14:50:00.970192 18169 raft_consensus.cc:388] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:50:00.970315 18169 raft_consensus.cc:482] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:50:00.970512 18169 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:50:00.972949 18169 raft_consensus.cc:504] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37446b885a44380ac241b882008e63a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44225 } }
I20260810 14:50:00.973417 18169 leader_election.cc:302] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d37446b885a44380ac241b882008e63a; no voters:
I20260810 14:50:00.973970 18169 leader_election.cc:288] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:50:00.974126 18182 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:50:00.974788 18182 raft_consensus.cc:686] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [term 1 LEADER]: Becoming Leader. State: Replica: d37446b885a44380ac241b882008e63a, State: Running, Role: LEADER
I20260810 14:50:00.975419 18182 consensus_queue.cc:237] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37446b885a44380ac241b882008e63a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44225 } }
I20260810 14:50:00.976091 18169 sys_catalog.cc:564] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:50:00.978509 18184 sys_catalog.cc:455] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [sys.catalog]: SysCatalogTable state changed. Reason: New leader d37446b885a44380ac241b882008e63a. Latest consensus state: current_term: 1 leader_uuid: "d37446b885a44380ac241b882008e63a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37446b885a44380ac241b882008e63a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44225 } } }
I20260810 14:50:00.978592 18183 sys_catalog.cc:455] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d37446b885a44380ac241b882008e63a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d37446b885a44380ac241b882008e63a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44225 } } }
I20260810 14:50:00.978986 18184 sys_catalog.cc:458] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:00.979116 18183 sys_catalog.cc:458] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:00.983911 18188 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:50:00.989223 18188 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:50:00.998433 18188 catalog_manager.cc:1347] Generated new cluster ID: f56d1629ebdf42e985c9b0574de6f1e5
I20260810 14:50:00.998565 18188 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:50:01.022254 18188 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:50:01.022719 18188 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:50:01.041895 18188 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d37446b885a44380ac241b882008e63a: Generated new TSK 0
I20260810 14:50:01.042778 18188 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:50:01.057713 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 18096
W20260810 14:50:01.059356 18164 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.30:53471
W20260810 14:50:01.060158 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 18096: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:50:01.060313 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 18096
2026-08-10T14:50:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2135 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:50:01 dist-test-slave-vkjp krb5kdc[18205](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:50:01 dist-test-slave-vkjp krb5kdc[18205](info): set up 2 sockets
Aug 10 14:50:01 dist-test-slave-vkjp krb5kdc[18205](info): commencing operation
krb5kdc: starting...
W20260810 14:50:02.274662 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.001s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:50:02 dist-test-slave-vkjp krb5kdc[18205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:50:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:50:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:50:02.594033 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40699
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40699
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:50:02.944687 18220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:50:02.944983 18220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:50:02.945078 18220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:50:02.952432 18220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:50:02.952569 18220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:50:02.952620 18220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:50:02.952685 18220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:50:02.952730 18220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:50:02.962965 18220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40699
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:50:02.964471 18220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:50:02.965834 18220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:50:02.978001 18227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:02.978197 18226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:02.978238 18230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:50:02.979166 18220 server_base.cc:1034] running on GCE node
I20260810 14:50:02.980067 18220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:50:02.981521 18220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:50:02.982769 18220 hybrid_clock.cc:648] HybridClock initialized: now 1786373402982731 us; error 53 us; skew 500 ppm
I20260810 14:50:02.983210 18220 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:50:02 dist-test-slave-vkjp krb5kdc[18205](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373402, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:50:02.989836 18220 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:50:02.991618 18220 webserver.cc:460] Webserver started at http://127.0.0.1:43603/ using document root <none> and password file <none>
I20260810 14:50:02.992302 18220 fs_manager.cc:359] Metadata directory not provided
I20260810 14:50:02.992411 18220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:50:02.992707 18220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:50:02.995940 18220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "531ce85dd5214ab5b7fd49c57a573df8"
format_stamp: "Formatted at 2026-08-10 14:50:02 on dist-test-slave-vkjp"
server_key: "c0f3b01faf98505f935fa9db77102aca"
server_key_iv: "8eba24b6693bcbe655a20a8b65f19a19"
server_key_version: "encryptionkey@0"
I20260810 14:50:02.996711 18220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "531ce85dd5214ab5b7fd49c57a573df8"
format_stamp: "Formatted at 2026-08-10 14:50:02 on dist-test-slave-vkjp"
server_key: "c0f3b01faf98505f935fa9db77102aca"
server_key_iv: "8eba24b6693bcbe655a20a8b65f19a19"
server_key_version: "encryptionkey@0"
I20260810 14:50:03.002153 18220 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:50:03.005791 18238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:50:03.007344 18220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:50:03.007565 18220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "531ce85dd5214ab5b7fd49c57a573df8"
format_stamp: "Formatted at 2026-08-10 14:50:02 on dist-test-slave-vkjp"
server_key: "c0f3b01faf98505f935fa9db77102aca"
server_key_iv: "8eba24b6693bcbe655a20a8b65f19a19"
server_key_version: "encryptionkey@0"
I20260810 14:50:03.007776 18220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:50:03.022735 18220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:50:03.027242 18220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:50:03.027558 18220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:50:03.046914 18220 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40699
I20260810 14:50:03.046945 18293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40699 every 8 connection(s)
I20260810 14:50:03.048386 18220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:50:03.052237 18294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:50:03.055778 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 18220
I20260810 14:50:03.055975 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:50:03.056552 25438 external_mini_cluster.cc:1428] Setting key ead99a3585b27a75b97583f15d3a00e0
I20260810 14:50:03.065726 18294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8: Bootstrap starting.
I20260810 14:50:03.069581 18294 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8: Neither blocks nor log segments found. Creating new log.
I20260810 14:50:03.070631 18294 log.cc:824] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8: Log is configured to *not* fsync() on all Append() calls
Aug 10 14:50:03 dist-test-slave-vkjp krb5kdc[18205](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373402, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260810 14:50:03.073952 18294 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8: No bootstrap required, opened a new log
I20260810 14:50:03.081112 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:03.058563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:60827 (local address 127.0.0.1:40699)
0810 14:50:03.059411 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:03.059424 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:03.059472 (+ 48us) server_negotiation.cc:407] Connection header received
0810 14:50:03.061320 (+ 1848us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:03.061353 (+ 33us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:03.061936 (+ 583us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:03.062385 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:03.063955 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:03.065978 (+ 2023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:03.067360 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:03.068457 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:03.068645 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:03.073197 (+ 4552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:03.073205 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:03.073220 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:03.073258 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:03.076566 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:03.077557 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:03.077563 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:03.077573 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:03.077746 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:03.078262 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:03.078266 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:03.078267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:03.078804 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:03.079059 (+ 255us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:03.079635 (+ 576us) server_negotiation.cc:299] Negotiation successful
0810 14:50:03.080010 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":626,"thread_start_us":334,"threads_started":1}
I20260810 14:50:03.083300 18294 raft_consensus.cc:348] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "531ce85dd5214ab5b7fd49c57a573df8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40699 } }
I20260810 14:50:03.083784 18294 raft_consensus.cc:374] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:50:03.083956 18294 raft_consensus.cc:729] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 531ce85dd5214ab5b7fd49c57a573df8, State: Initialized, Role: FOLLOWER
I20260810 14:50:03.084596 18294 consensus_queue.cc:260] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "531ce85dd5214ab5b7fd49c57a573df8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40699 } }
I20260810 14:50:03.084872 18294 raft_consensus.cc:388] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:50:03.085022 18294 raft_consensus.cc:482] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:50:03.085217 18294 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:50:03.085543 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:50:03.088500 18294 raft_consensus.cc:504] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "531ce85dd5214ab5b7fd49c57a573df8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40699 } }
I20260810 14:50:03.089156 18294 leader_election.cc:302] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 531ce85dd5214ab5b7fd49c57a573df8; no voters:
I20260810 14:50:03.089946 18294 leader_election.cc:288] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:50:03.090106 18304 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:50:03.092186 18304 raft_consensus.cc:686] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [term 1 LEADER]: Becoming Leader. State: Replica: 531ce85dd5214ab5b7fd49c57a573df8, State: Running, Role: LEADER
I20260810 14:50:03.092840 18304 consensus_queue.cc:237] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "531ce85dd5214ab5b7fd49c57a573df8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40699 } }
I20260810 14:50:03.093237 18294 sys_catalog.cc:564] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:50:03.096191 18307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 531ce85dd5214ab5b7fd49c57a573df8. Latest consensus state: current_term: 1 leader_uuid: "531ce85dd5214ab5b7fd49c57a573df8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "531ce85dd5214ab5b7fd49c57a573df8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40699 } } }
I20260810 14:50:03.096748 18307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:03.097479 18306 sys_catalog.cc:455] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "531ce85dd5214ab5b7fd49c57a573df8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "531ce85dd5214ab5b7fd49c57a573df8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40699 } } }
I20260810 14:50:03.097909 18306 sys_catalog.cc:458] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:03.100996 18314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:50:03.109328 18314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:50:03.111534 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:03.092194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:36671 (local address 127.0.0.1:40699)
0810 14:50:03.092391 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:03.092397 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:03.092475 (+ 78us) server_negotiation.cc:407] Connection header received
0810 14:50:03.092726 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:03.092732 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:03.092815 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:03.092965 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:03.094891 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:03.096285 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:03.098268 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:03.099172 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:03.099297 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:03.103378 (+ 4081us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:03.103383 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:03.103387 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:03.103451 (+ 64us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:03.107702 (+ 4251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:03.108714 (+ 1012us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:03.108720 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:03.108724 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:03.108879 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:03.109517 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:03.109522 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:03.109527 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:03.109969 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:03.110216 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:03.110931 (+ 715us) server_negotiation.cc:299] Negotiation successful
0810 14:50:03.111165 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 14:50:03.125447 18314 catalog_manager.cc:1347] Generated new cluster ID: ca223d68b1854c9eb0b24c34a7a3a456
I20260810 14:50:03.125566 18314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:50:03.161522 18314 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:50:03.162881 18314 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:50:03.171123 18314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 531ce85dd5214ab5b7fd49c57a573df8: Generated new TSK 0
I20260810 14:50:03.172029 18314 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:50:03.182837 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 18220
I20260810 14:50:03.183960 18289 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:50:03.277524 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 18220
W20260810 14:50:03.300405 18203 connection.cc:535] client connection to 127.0.0.1:40699 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40699): Connection reset by peer (error 104)
2026-08-10T14:50:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2222 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:50:03 dist-test-slave-vkjp krb5kdc[18331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:50:03 dist-test-slave-vkjp krb5kdc[18331](info): set up 2 sockets
Aug 10 14:50:03 dist-test-slave-vkjp krb5kdc[18331](info): commencing operation
krb5kdc: starting...
W20260810 14:50:04.494894 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.000s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:50:04 dist-test-slave-vkjp krb5kdc[18331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373404, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:50:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:50:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:50:04.807740 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45659
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41697
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45659
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:50:05.153040 18344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:50:05.153335 18344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:50:05.153509 18344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:50:05.160889 18344 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:50:05.161003 18344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:50:05.161137 18344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:50:05.161248 18344 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:50:05.161309 18344 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:50:05.171243 18344 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41697
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45659
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45659
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:50:05.172575 18344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:50:05.173890 18344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:50:05.185915 18352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:05.186021 18349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:05.186060 18350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:50:05.186991 18344 server_base.cc:1034] running on GCE node
I20260810 14:50:05.187801 18344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:50:05.189287 18344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:50:05.190534 18344 hybrid_clock.cc:648] HybridClock initialized: now 1786373405190487 us; error 44 us; skew 500 ppm
I20260810 14:50:05.190900 18344 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:50:05 dist-test-slave-vkjp krb5kdc[18331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373405, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:50:05.194162 18344 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:50:05.195822 18344 webserver.cc:460] Webserver started at http://127.0.0.1:44301/ using document root <none> and password file <none>
I20260810 14:50:05.196525 18344 fs_manager.cc:359] Metadata directory not provided
I20260810 14:50:05.196678 18344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:50:05.196997 18344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:50:05.200093 18344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "bf9f573c74464db5b744b6ba5e13b3a1"
format_stamp: "Formatted at 2026-08-10 14:50:05 on dist-test-slave-vkjp"
server_key: "fb19d4488f078ba27efd9465b0da77db"
server_key_iv: "89cdd8869cb52ca90b5d629568629b0e"
server_key_version: "encryptionkey@0"
I20260810 14:50:05.200934 18344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "bf9f573c74464db5b744b6ba5e13b3a1"
format_stamp: "Formatted at 2026-08-10 14:50:05 on dist-test-slave-vkjp"
server_key: "fb19d4488f078ba27efd9465b0da77db"
server_key_iv: "89cdd8869cb52ca90b5d629568629b0e"
server_key_version: "encryptionkey@0"
I20260810 14:50:05.206743 18344 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260810 14:50:05.210486 18360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:50:05.211884 18344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 14:50:05.212082 18344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "bf9f573c74464db5b744b6ba5e13b3a1"
format_stamp: "Formatted at 2026-08-10 14:50:05 on dist-test-slave-vkjp"
server_key: "fb19d4488f078ba27efd9465b0da77db"
server_key_iv: "89cdd8869cb52ca90b5d629568629b0e"
server_key_version: "encryptionkey@0"
I20260810 14:50:05.212293 18344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:50:05.231489 18344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:50:05.235723 18344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:50:05.236033 18344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:50:05.255643 18344 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45659
I20260810 14:50:05.255677 18419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45659 every 8 connection(s)
I20260810 14:50:05.257112 18344 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:50:05.258677 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 18344
I20260810 14:50:05.258864 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:50:05.259495 25438 external_mini_cluster.cc:1428] Setting key d133fe62a52da18854d7be4f9af05df1
I20260810 14:50:05.261945 18420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:50:05.274020 18423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:05.262263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:44251 (local address 127.0.0.1:45659)
0810 14:50:05.263068 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:05.263123 (+ 55us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:05.263182 (+ 59us) server_negotiation.cc:407] Connection header received
0810 14:50:05.264781 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:05.264802 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:05.265189 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:05.265517 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:05.266157 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:05.267542 (+ 1385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:05.269467 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:05.270130 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:05.270275 (+ 145us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:05.271662 (+ 1387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:05.271670 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:05.271686 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:50:05.272249 (+ 563us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:05.272331 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:50:05.272358 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:05.272487 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:05.272835 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 14:50:05.273071 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":371,"threads_started":1}
I20260810 14:50:05.275497 18420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1: Bootstrap starting.
I20260810 14:50:05.279342 18420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1: Neither blocks nor log segments found. Creating new log.
I20260810 14:50:05.280829 18420 log.cc:824] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1: Log is configured to *not* fsync() on all Append() calls
I20260810 14:50:05.280879 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:50:05.284435 18420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1: No bootstrap required, opened a new log
I20260810 14:50:05.295670 18423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:05.289038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:55633 (local address 127.0.0.1:45659)
0810 14:50:05.289212 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:05.289217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:05.289233 (+ 16us) server_negotiation.cc:407] Connection header received
0810 14:50:05.289297 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:05.289301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:05.289362 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:05.289450 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:05.290097 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:05.290917 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:05.292911 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:05.293408 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:05.293516 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:05.294793 (+ 1277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:05.294798 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:05.294801 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:50:05.294820 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:05.294875 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:50:05.294891 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:05.294986 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:05.295237 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 14:50:05.295284 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260810 14:50:05.295849 18420 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9f573c74464db5b744b6ba5e13b3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45659 } }
I20260810 14:50:05.296373 18420 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:50:05.296509 18420 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf9f573c74464db5b744b6ba5e13b3a1, State: Initialized, Role: FOLLOWER
I20260810 14:50:05.297206 18420 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9f573c74464db5b744b6ba5e13b3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45659 } }
I20260810 14:50:05.297462 18420 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:50:05.297624 18420 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:50:05.297799 18420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:50:05.300103 18420 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9f573c74464db5b744b6ba5e13b3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45659 } }
I20260810 14:50:05.300637 18420 leader_election.cc:302] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf9f573c74464db5b744b6ba5e13b3a1; no voters:
I20260810 14:50:05.301232 18420 leader_election.cc:288] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:50:05.301391 18430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:50:05.302196 18430 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [term 1 LEADER]: Becoming Leader. State: Replica: bf9f573c74464db5b744b6ba5e13b3a1, State: Running, Role: LEADER
I20260810 14:50:05.302878 18430 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9f573c74464db5b744b6ba5e13b3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45659 } }
I20260810 14:50:05.303818 18420 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:50:05.305913 18431 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf9f573c74464db5b744b6ba5e13b3a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9f573c74464db5b744b6ba5e13b3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45659 } } }
I20260810 14:50:05.305944 18432 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf9f573c74464db5b744b6ba5e13b3a1. Latest consensus state: current_term: 1 leader_uuid: "bf9f573c74464db5b744b6ba5e13b3a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf9f573c74464db5b744b6ba5e13b3a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45659 } } }
I20260810 14:50:05.306378 18431 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:05.306378 18432 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:05.308018 18436 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:50:05.313627 18436 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:50:05.323660 18436 catalog_manager.cc:1347] Generated new cluster ID: 557f7f17d3f842e290ae189117f8d25d
I20260810 14:50:05.323791 18436 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:50:05.349236 18436 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:50:05.350557 18436 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:50:05.364621 18436 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf9f573c74464db5b744b6ba5e13b3a1: Generated new TSK 0
I20260810 14:50:05.365556 18436 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:50:05.373365 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 18344
W20260810 14:50:05.374559 18415 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.30:44251
W20260810 14:50:05.375396 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 18344: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:50:05.375569 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 18344
W20260810 14:50:05.397802 18329 connection.cc:535] client connection to 127.0.0.1:45659 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45659): Connection reset by peer (error 104)
2026-08-10T14:50:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2092 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:50:05 dist-test-slave-vkjp krb5kdc[18452](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:50:05 dist-test-slave-vkjp krb5kdc[18452](info): set up 2 sockets
Aug 10 14:50:05 dist-test-slave-vkjp krb5kdc[18452](info): commencing operation
krb5kdc: starting...
W20260810 14:50:06.585880 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:50:06 dist-test-slave-vkjp krb5kdc[18452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373406, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:50:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:50:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260810 14:50:06.901935 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39909
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39271
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:50:07.242787 18465 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:50:07.243152 18465 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:50:07.243335 18465 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:50:07.250916 18465 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:50:07.251062 18465 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:50:07.251171 18465 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:50:07.251264 18465 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:50:07.251359 18465 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:50:07.261404 18465 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39271
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39909
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:50:07.262797 18465 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:50:07.264171 18465 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:50:07.276618 18472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:07.276531 18473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:07.276589 18475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:50:07.277725 18465 server_base.cc:1034] running on GCE node
I20260810 14:50:07.278554 18465 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:50:07.280097 18465 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:50:07.281308 18465 hybrid_clock.cc:648] HybridClock initialized: now 1786373407281264 us; error 31 us; skew 500 ppm
I20260810 14:50:07.281667 18465 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:50:07 dist-test-slave-vkjp krb5kdc[18452](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373407, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:50:07.284816 18465 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260810 14:50:07.286334 18465 webserver.cc:460] Webserver started at http://127.0.0.1:36379/ using document root <none> and password file <none>
I20260810 14:50:07.287014 18465 fs_manager.cc:359] Metadata directory not provided
I20260810 14:50:07.287204 18465 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:50:07.287529 18465 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:50:07.290483 18465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "25b1a485b7f84877a5c05c82f1c69906"
format_stamp: "Formatted at 2026-08-10 14:50:07 on dist-test-slave-vkjp"
server_key: "3b34f478b9f85a56b5be733c8fe496b5"
server_key_iv: "916fdde91c6d3372d98dfd8f65f1f9d1"
server_key_version: "encryptionkey@0"
I20260810 14:50:07.291309 18465 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "25b1a485b7f84877a5c05c82f1c69906"
format_stamp: "Formatted at 2026-08-10 14:50:07 on dist-test-slave-vkjp"
server_key: "3b34f478b9f85a56b5be733c8fe496b5"
server_key_iv: "916fdde91c6d3372d98dfd8f65f1f9d1"
server_key_version: "encryptionkey@0"
I20260810 14:50:07.296525 18465 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:50:07.299896 18482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:50:07.301237 18465 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:50:07.301419 18465 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "25b1a485b7f84877a5c05c82f1c69906"
format_stamp: "Formatted at 2026-08-10 14:50:07 on dist-test-slave-vkjp"
server_key: "3b34f478b9f85a56b5be733c8fe496b5"
server_key_iv: "916fdde91c6d3372d98dfd8f65f1f9d1"
server_key_version: "encryptionkey@0"
I20260810 14:50:07.301620 18465 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:50:07.307565 18465 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:50:07.311779 18465 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:50:07.312098 18465 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:50:07.331393 18465 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39909
I20260810 14:50:07.331410 18541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39909 every 8 connection(s)
I20260810 14:50:07.332973 18465 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:50:07.336851 18542 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:50:07.342262 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 18465
I20260810 14:50:07.342437 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:50:07.342983 25438 external_mini_cluster.cc:1428] Setting key 111ede5293d2707c9f945916a5cebc9f
I20260810 14:50:07.348006 18542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906: Bootstrap starting.
I20260810 14:50:07.350677 18545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:07.344910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:49025 (local address 127.0.0.1:39909)
0810 14:50:07.345706 (+ 796us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:07.345718 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:07.345760 (+ 42us) server_negotiation.cc:407] Connection header received
0810 14:50:07.347463 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:07.347486 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:07.347793 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:07.348106 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:07.349154 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:07.349160 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:07.349172 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:50:07.349186 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:07.349244 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:50:07.349263 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:07.349426 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:07.349554 (+ 128us) server_negotiation.cc:299] Negotiation successful
0810 14:50:07.349758 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":371,"threads_started":1}
I20260810 14:50:07.351671 18542 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906: Neither blocks nor log segments found. Creating new log.
I20260810 14:50:07.353109 18542 log.cc:824] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906: Log is configured to *not* fsync() on all Append() calls
I20260810 14:50:07.354893 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:50:07.356762 18542 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906: No bootstrap required, opened a new log
I20260810 14:50:07.363658 18545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:07.360518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.30:55759 (local address 127.0.0.1:39909)
0810 14:50:07.360785 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:07.360799 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:07.361483 (+ 684us) server_negotiation.cc:407] Connection header received
0810 14:50:07.361665 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:07.361669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:07.361719 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:07.361806 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:07.362898 (+ 1092us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:07.362902 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:07.362905 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:50:07.362923 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:07.362964 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:50:07.362978 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:07.363072 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:07.363288 (+ 216us) server_negotiation.cc:299] Negotiation successful
0810 14:50:07.363340 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260810 14:50:07.367360 18542 raft_consensus.cc:348] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b1a485b7f84877a5c05c82f1c69906" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39909 } }
I20260810 14:50:07.367781 18542 raft_consensus.cc:374] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:50:07.367928 18542 raft_consensus.cc:729] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25b1a485b7f84877a5c05c82f1c69906, State: Initialized, Role: FOLLOWER
I20260810 14:50:07.368677 18542 consensus_queue.cc:260] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b1a485b7f84877a5c05c82f1c69906" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39909 } }
I20260810 14:50:07.369045 18542 raft_consensus.cc:388] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:50:07.369243 18542 raft_consensus.cc:482] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:50:07.369467 18542 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:50:07.372177 18542 raft_consensus.cc:504] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b1a485b7f84877a5c05c82f1c69906" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39909 } }
I20260810 14:50:07.372814 18542 leader_election.cc:302] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 25b1a485b7f84877a5c05c82f1c69906; no voters:
I20260810 14:50:07.373508 18542 leader_election.cc:288] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:50:07.373615 18553 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:50:07.374372 18553 raft_consensus.cc:686] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [term 1 LEADER]: Becoming Leader. State: Replica: 25b1a485b7f84877a5c05c82f1c69906, State: Running, Role: LEADER
I20260810 14:50:07.374991 18553 consensus_queue.cc:237] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b1a485b7f84877a5c05c82f1c69906" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39909 } }
I20260810 14:50:07.375946 18542 sys_catalog.cc:564] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:50:07.378160 18554 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "25b1a485b7f84877a5c05c82f1c69906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b1a485b7f84877a5c05c82f1c69906" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39909 } } }
I20260810 14:50:07.378228 18555 sys_catalog.cc:455] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 25b1a485b7f84877a5c05c82f1c69906. Latest consensus state: current_term: 1 leader_uuid: "25b1a485b7f84877a5c05c82f1c69906" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "25b1a485b7f84877a5c05c82f1c69906" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39909 } } }
I20260810 14:50:07.378623 18554 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:07.378669 18555 sys_catalog.cc:458] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:07.383253 18558 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:50:07.388979 18558 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:50:07.399355 18558 catalog_manager.cc:1347] Generated new cluster ID: f40e847378ce44f9b512164f9f296e05
I20260810 14:50:07.399473 18558 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:50:07.428707 18558 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:50:07.429159 18558 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:50:07.438982 18558 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 25b1a485b7f84877a5c05c82f1c69906: Generated new TSK 0
I20260810 14:50:07.439872 18558 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:50:07.447044 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 18465
W20260810 14:50:07.448578 18535 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.30:49025
W20260810 14:50:07.449324 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 18465: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:50:07.449498 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 18465
2026-08-10T14:50:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2074 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:50:07 dist-test-slave-vkjp krb5kdc[18578](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:50:07 dist-test-slave-vkjp krb5kdc[18578](info): set up 2 sockets
Aug 10 14:50:07 dist-test-slave-vkjp krb5kdc[18578](info): commencing operation
krb5kdc: starting...
W20260810 14:50:08.660935 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.002s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:50:08 dist-test-slave-vkjp krb5kdc[18578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:50:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:50:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:50:08.984534 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:39245
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:39101
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:39245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:50:09.332871 18591 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:50:09.333217 18591 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:50:09.333412 18591 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:50:09.340823 18591 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:50:09.340981 18591 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:50:09.341110 18591 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:50:09.341225 18591 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:50:09.341315 18591 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:50:09.351063 18591 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:39101
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:39245
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:39245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:50:09.352416 18591 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:50:09.353747 18591 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:50:09.366204 18597 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:09.366196 18596 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:09.366199 18600 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:50:09.367254 18591 server_base.cc:1034] running on GCE node
I20260810 14:50:09.368058 18591 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:50:09.369578 18591 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:50:09.370831 18591 hybrid_clock.cc:648] HybridClock initialized: now 1786373409370765 us; error 63 us; skew 500 ppm
I20260810 14:50:09.371243 18591 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:50:09 dist-test-slave-vkjp krb5kdc[18578](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373409, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:50:09.379359 18591 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:50:09.380862 18591 webserver.cc:460] Webserver started at http://127.24.215.190:35273/ using document root <none> and password file <none>
I20260810 14:50:09.381614 18591 fs_manager.cc:359] Metadata directory not provided
I20260810 14:50:09.381770 18591 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:50:09.382100 18591 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:50:09.385212 18591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "a704e594bcbf4445b2b9f3f620605649"
format_stamp: "Formatted at 2026-08-10 14:50:09 on dist-test-slave-vkjp"
server_key: "7cb0ab1d280f885e208a3e8a12acde21"
server_key_iv: "d26e9a08158ad5943e01a5b858e7a80d"
server_key_version: "encryptionkey@0"
I20260810 14:50:09.385990 18591 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "a704e594bcbf4445b2b9f3f620605649"
format_stamp: "Formatted at 2026-08-10 14:50:09 on dist-test-slave-vkjp"
server_key: "7cb0ab1d280f885e208a3e8a12acde21"
server_key_iv: "d26e9a08158ad5943e01a5b858e7a80d"
server_key_version: "encryptionkey@0"
I20260810 14:50:09.391240 18591 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260810 14:50:09.394556 18607 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:50:09.395890 18591 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:50:09.396081 18591 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "a704e594bcbf4445b2b9f3f620605649"
format_stamp: "Formatted at 2026-08-10 14:50:09 on dist-test-slave-vkjp"
server_key: "7cb0ab1d280f885e208a3e8a12acde21"
server_key_iv: "d26e9a08158ad5943e01a5b858e7a80d"
server_key_version: "encryptionkey@0"
I20260810 14:50:09.396286 18591 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:50:09.428187 18591 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:50:09.433177 18591 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:50:09.433501 18591 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:50:09.452986 18591 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:39245
I20260810 14:50:09.453071 18664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:39245 every 8 connection(s)
I20260810 14:50:09.454351 18591 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:50:09.457508 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 18591
I20260810 14:50:09.457695 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:50:09.458292 25438 external_mini_cluster.cc:1428] Setting key 569a81370225a2740aa014a03886f40b
I20260810 14:50:09.458456 18665 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:50:09.473043 18665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649: Bootstrap starting.
Aug 10 14:50:09 dist-test-slave-vkjp krb5kdc[18578](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373408, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:50:09.476939 18665 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649: Neither blocks nor log segments found. Creating new log.
I20260810 14:50:09.478340 18665 log.cc:824] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649: Log is configured to *not* fsync() on all Append() calls
I20260810 14:50:09.481972 18665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649: No bootstrap required, opened a new log
I20260810 14:50:09.483929 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:09.460110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49985 (local address 127.24.215.190:39245)
0810 14:50:09.460916 (+ 806us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:09.460928 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:09.460976 (+ 48us) server_negotiation.cc:407] Connection header received
0810 14:50:09.463045 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:09.463066 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:09.463521 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:09.463835 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:09.465683 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:09.466973 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:09.468824 (+ 1851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:09.469510 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:09.469646 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:09.474729 (+ 5083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:09.474749 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:09.474770 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:09.474835 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:09.479064 (+ 4229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:09.480131 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:09.480139 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:09.480147 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:09.480356 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:09.480810 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:09.480815 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:09.480818 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:09.481507 (+ 689us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:09.481859 (+ 352us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:09.482207 (+ 348us) server_negotiation.cc:299] Negotiation successful
0810 14:50:09.482575 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":391,"threads_started":1}
I20260810 14:50:09.487965 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:50:09.490964 18665 raft_consensus.cc:348] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a704e594bcbf4445b2b9f3f620605649" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39245 } }
I20260810 14:50:09.491415 18665 raft_consensus.cc:374] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:50:09.491537 18665 raft_consensus.cc:729] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a704e594bcbf4445b2b9f3f620605649, State: Initialized, Role: FOLLOWER
I20260810 14:50:09.492174 18665 consensus_queue.cc:260] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a704e594bcbf4445b2b9f3f620605649" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39245 } }
I20260810 14:50:09.492472 18665 raft_consensus.cc:388] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:50:09.492640 18665 raft_consensus.cc:482] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:50:09.492847 18665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:50:09.496088 18665 raft_consensus.cc:504] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a704e594bcbf4445b2b9f3f620605649" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39245 } }
I20260810 14:50:09.496716 18665 leader_election.cc:302] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a704e594bcbf4445b2b9f3f620605649; no voters:
I20260810 14:50:09.497368 18665 leader_election.cc:288] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:50:09.497488 18675 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:50:09.499774 18665 sys_catalog.cc:564] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:50:09.503472 18675 raft_consensus.cc:686] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [term 1 LEADER]: Becoming Leader. State: Replica: a704e594bcbf4445b2b9f3f620605649, State: Running, Role: LEADER
I20260810 14:50:09.504232 18675 consensus_queue.cc:237] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a704e594bcbf4445b2b9f3f620605649" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39245 } }
I20260810 14:50:09.508872 18676 sys_catalog.cc:455] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a704e594bcbf4445b2b9f3f620605649" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a704e594bcbf4445b2b9f3f620605649" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39245 } } }
I20260810 14:50:09.508912 18677 sys_catalog.cc:455] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a704e594bcbf4445b2b9f3f620605649. Latest consensus state: current_term: 1 leader_uuid: "a704e594bcbf4445b2b9f3f620605649" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a704e594bcbf4445b2b9f3f620605649" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39245 } } }
I20260810 14:50:09.509341 18677 sys_catalog.cc:458] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:09.509338 18676 sys_catalog.cc:458] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:09.512125 18691 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:50:09.513980 18668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:09.492362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55547 (local address 127.24.215.190:39245)
0810 14:50:09.492578 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:09.492587 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:09.493133 (+ 546us) server_negotiation.cc:407] Connection header received
0810 14:50:09.493312 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:09.493316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:09.493377 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:09.493467 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:09.495384 (+ 1917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:09.496129 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:09.497999 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:09.498653 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:09.498767 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:09.502916 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:09.502926 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:09.502935 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:09.502987 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:09.510092 (+ 7105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:09.510999 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:09.511003 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:09.511005 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:09.511180 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:09.511854 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:09.511859 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:09.511862 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:09.512381 (+ 519us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:09.512667 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:09.513428 (+ 761us) server_negotiation.cc:299] Negotiation successful
0810 14:50:09.513621 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260810 14:50:09.516924 18691 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:50:09.524813 18691 catalog_manager.cc:1347] Generated new cluster ID: 8710546e5dc0420eba094b90556ace62
I20260810 14:50:09.524922 18691 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:50:09.551501 18691 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:50:09.552850 18691 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:50:09.563323 18691 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a704e594bcbf4445b2b9f3f620605649: Generated new TSK 0
I20260810 14:50:09.564204 18691 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:50:09.574059 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 18591
I20260810 14:50:09.575238 18659 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:50:09.672127 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 18591
W20260810 14:50:09.694029 18576 connection.cc:535] client connection to 127.24.215.190:39245 recv error: Network error: failed to read from TLS socket (remote: 127.24.215.190:39245): Connection reset by peer (error 104)
2026-08-10T14:50:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2221 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:50:09 dist-test-slave-vkjp krb5kdc[18700](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:50:09 dist-test-slave-vkjp krb5kdc[18700](info): set up 2 sockets
Aug 10 14:50:09 dist-test-slave-vkjp krb5kdc[18700](info): commencing operation
krb5kdc: starting...
W20260810 14:50:10.901890 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.189s user 0.003s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:50:11 dist-test-slave-vkjp krb5kdc[18700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373411, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:50:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:50:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:50:11.218540 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:43963
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:34167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:43963
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:50:11.567715 18714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:50:11.568079 18714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:50:11.568259 18714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:50:11.576076 18714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:50:11.576181 18714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:50:11.576226 18714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:50:11.576287 18714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:50:11.576335 18714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:50:11.586166 18714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:34167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:43963
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:43963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:50:11.587951 18714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:50:11.589882 18714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:50:11.603956 18722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:11.604038 18721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:11.604013 18724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:50:11.604816 18714 server_base.cc:1034] running on GCE node
I20260810 14:50:11.605561 18714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:50:11.607000 18714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:50:11.608233 18714 hybrid_clock.cc:648] HybridClock initialized: now 1786373411608179 us; error 39 us; skew 500 ppm
I20260810 14:50:11.608649 18714 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:50:11 dist-test-slave-vkjp krb5kdc[18700](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373411, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:50:11.611877 18714 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:50:11.613380 18714 webserver.cc:460] Webserver started at http://127.24.215.190:44457/ using document root <none> and password file <none>
I20260810 14:50:11.614058 18714 fs_manager.cc:359] Metadata directory not provided
I20260810 14:50:11.614192 18714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:50:11.614518 18714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:50:11.617573 18714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "efcca4b8a79443a6ba4a66b66cebf027"
format_stamp: "Formatted at 2026-08-10 14:50:11 on dist-test-slave-vkjp"
server_key: "9ccd9bd107185c25e9401774fc5a13b0"
server_key_iv: "7dcf4ef6d254803817ed6c5a1b906a1a"
server_key_version: "encryptionkey@0"
I20260810 14:50:11.618336 18714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "efcca4b8a79443a6ba4a66b66cebf027"
format_stamp: "Formatted at 2026-08-10 14:50:11 on dist-test-slave-vkjp"
server_key: "9ccd9bd107185c25e9401774fc5a13b0"
server_key_iv: "7dcf4ef6d254803817ed6c5a1b906a1a"
server_key_version: "encryptionkey@0"
I20260810 14:50:11.623587 18714 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260810 14:50:11.626952 18733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:50:11.628296 18714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:50:11.628500 18714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "efcca4b8a79443a6ba4a66b66cebf027"
format_stamp: "Formatted at 2026-08-10 14:50:11 on dist-test-slave-vkjp"
server_key: "9ccd9bd107185c25e9401774fc5a13b0"
server_key_iv: "7dcf4ef6d254803817ed6c5a1b906a1a"
server_key_version: "encryptionkey@0"
I20260810 14:50:11.628705 18714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:50:11.646855 18714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:50:11.650892 18714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:50:11.651270 18714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:50:11.670240 18714 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:43963
I20260810 14:50:11.670266 18788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:43963 every 8 connection(s)
I20260810 14:50:11.671725 18714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:50:11.675508 18789 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:50:11.679994 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 18714
I20260810 14:50:11.680184 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:50:11.680729 25438 external_mini_cluster.cc:1428] Setting key b6e7b1fb2d32760fc36a3d5ed670399a
I20260810 14:50:11.689038 18789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027: Bootstrap starting.
I20260810 14:50:11.692766 18789 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027: Neither blocks nor log segments found. Creating new log.
I20260810 14:50:11.693884 18789 log.cc:824] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027: Log is configured to *not* fsync() on all Append() calls
I20260810 14:50:11.694044 18792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:11.682502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37227 (local address 127.24.215.190:43963)
0810 14:50:11.683349 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:11.683361 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:11.683401 (+ 40us) server_negotiation.cc:407] Connection header received
0810 14:50:11.684969 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:11.684989 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:11.685403 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:11.685731 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:11.686326 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:11.687749 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:11.689627 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:11.690260 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:11.690395 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:11.691773 (+ 1378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:11.691781 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:11.691793 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:50:11.692294 (+ 501us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:11.692365 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:50:11.692391 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:11.692508 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:11.692877 (+ 369us) server_negotiation.cc:299] Negotiation successful
0810 14:50:11.693072 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":619,"thread_start_us":392,"threads_started":1}
I20260810 14:50:11.696895 18789 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027: No bootstrap required, opened a new log
I20260810 14:50:11.697969 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:50:11.705085 18789 raft_consensus.cc:348] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efcca4b8a79443a6ba4a66b66cebf027" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 43963 } }
I20260810 14:50:11.705540 18789 raft_consensus.cc:374] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:50:11.705662 18789 raft_consensus.cc:729] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efcca4b8a79443a6ba4a66b66cebf027, State: Initialized, Role: FOLLOWER
I20260810 14:50:11.706254 18789 consensus_queue.cc:260] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efcca4b8a79443a6ba4a66b66cebf027" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 43963 } }
I20260810 14:50:11.706496 18789 raft_consensus.cc:388] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:50:11.706651 18789 raft_consensus.cc:482] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:50:11.706828 18789 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:50:11.710184 18792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:11.702390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44571 (local address 127.24.215.190:43963)
0810 14:50:11.702588 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:11.702594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:11.703639 (+ 1045us) server_negotiation.cc:407] Connection header received
0810 14:50:11.703801 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:11.703809 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:11.703888 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:11.703999 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:11.704501 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:11.705614 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:11.707365 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:11.708114 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:11.708247 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:11.709323 (+ 1076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:11.709330 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:11.709334 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:50:11.709355 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:11.709400 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:50:11.709419 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:11.709535 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:11.709752 (+ 217us) server_negotiation.cc:299] Negotiation successful
0810 14:50:11.709805 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260810 14:50:11.710182 18789 raft_consensus.cc:504] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efcca4b8a79443a6ba4a66b66cebf027" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 43963 } }
I20260810 14:50:11.710750 18789 leader_election.cc:302] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efcca4b8a79443a6ba4a66b66cebf027; no voters:
I20260810 14:50:11.711498 18789 leader_election.cc:288] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:50:11.711652 18799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:50:11.712368 18799 raft_consensus.cc:686] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [term 1 LEADER]: Becoming Leader. State: Replica: efcca4b8a79443a6ba4a66b66cebf027, State: Running, Role: LEADER
I20260810 14:50:11.712968 18799 consensus_queue.cc:237] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efcca4b8a79443a6ba4a66b66cebf027" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 43963 } }
I20260810 14:50:11.713755 18789 sys_catalog.cc:564] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:50:11.716003 18800 sys_catalog.cc:455] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efcca4b8a79443a6ba4a66b66cebf027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efcca4b8a79443a6ba4a66b66cebf027" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 43963 } } }
I20260810 14:50:11.716377 18800 sys_catalog.cc:458] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:11.719488 18803 sys_catalog.cc:455] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efcca4b8a79443a6ba4a66b66cebf027. Latest consensus state: current_term: 1 leader_uuid: "efcca4b8a79443a6ba4a66b66cebf027" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efcca4b8a79443a6ba4a66b66cebf027" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 43963 } } }
I20260810 14:50:11.719887 18803 sys_catalog.cc:458] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:11.720491 18807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:50:11.727403 18807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:50:11.737692 18807 catalog_manager.cc:1347] Generated new cluster ID: e4f932df398043d69f49d6bcc489e411
I20260810 14:50:11.737821 18807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:50:11.765430 18807 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:50:11.766744 18807 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:50:11.773190 18807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efcca4b8a79443a6ba4a66b66cebf027: Generated new TSK 0
I20260810 14:50:11.774034 18807 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:50:11.790760 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 18714
W20260810 14:50:11.792367 18782 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:37227
W20260810 14:50:11.793149 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 18714: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:50:11.793316 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 18714
W20260810 14:50:11.815306 18698 connection.cc:535] client connection to 127.24.215.190:43963 recv error: Network error: failed to read from TLS socket (remote: 127.24.215.190:43963): Connection reset by peer (error 104)
2026-08-10T14:50:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2121 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:50:11 dist-test-slave-vkjp krb5kdc[18824](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:50:11 dist-test-slave-vkjp krb5kdc[18824](info): set up 2 sockets
Aug 10 14:50:11 dist-test-slave-vkjp krb5kdc[18824](info): commencing operation
krb5kdc: starting...
W20260810 14:50:13.007011 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:50:13 dist-test-slave-vkjp krb5kdc[18824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373413, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:50:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:50:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:50:13.321906 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:40535
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:40851
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:40535
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:50:13.669159 18839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:50:13.669474 18839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:50:13.669648 18839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:50:13.676751 18839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:50:13.676889 18839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:50:13.677000 18839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:50:13.677103 18839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:50:13.677186 18839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:50:13.686874 18839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:40851
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:40535
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:40535
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:50:13.688205 18839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:50:13.689550 18839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:50:13.701540 18844 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:13.701565 18846 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:13.701604 18848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:50:13.702318 18839 server_base.cc:1034] running on GCE node
I20260810 14:50:13.703215 18839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:50:13.704612 18839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:50:13.705864 18839 hybrid_clock.cc:648] HybridClock initialized: now 1786373413705822 us; error 50 us; skew 500 ppm
I20260810 14:50:13.706251 18839 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:50:13 dist-test-slave-vkjp krb5kdc[18824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373413, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:50:13.709896 18839 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:50:13.711505 18839 webserver.cc:460] Webserver started at http://127.24.215.190:45923/ using document root <none> and password file <none>
I20260810 14:50:13.712188 18839 fs_manager.cc:359] Metadata directory not provided
I20260810 14:50:13.712340 18839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:50:13.712693 18839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:50:13.715776 18839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "ed09e9e9f5434756b6ef43a1df75764c"
format_stamp: "Formatted at 2026-08-10 14:50:13 on dist-test-slave-vkjp"
server_key: "a3299e5d20f81f922eecb402b9a047a5"
server_key_iv: "65b82d8988cad441fe240a191624e5f7"
server_key_version: "encryptionkey@0"
I20260810 14:50:13.716566 18839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "ed09e9e9f5434756b6ef43a1df75764c"
format_stamp: "Formatted at 2026-08-10 14:50:13 on dist-test-slave-vkjp"
server_key: "a3299e5d20f81f922eecb402b9a047a5"
server_key_iv: "65b82d8988cad441fe240a191624e5f7"
server_key_version: "encryptionkey@0"
I20260810 14:50:13.721733 18839 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260810 14:50:13.725073 18855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:50:13.726292 18839 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260810 14:50:13.726485 18839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "ed09e9e9f5434756b6ef43a1df75764c"
format_stamp: "Formatted at 2026-08-10 14:50:13 on dist-test-slave-vkjp"
server_key: "a3299e5d20f81f922eecb402b9a047a5"
server_key_iv: "65b82d8988cad441fe240a191624e5f7"
server_key_version: "encryptionkey@0"
I20260810 14:50:13.726691 18839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:50:13.732383 18839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:50:13.736255 18839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:50:13.736560 18839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:50:13.756556 18839 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:40535
I20260810 14:50:13.756623 18915 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:40535 every 8 connection(s)
I20260810 14:50:13.758061 18839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:50:13.761004 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 18839
I20260810 14:50:13.761227 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:50:13.761758 25438 external_mini_cluster.cc:1428] Setting key 8903b4770ad235b804c69e28938a6d8f
I20260810 14:50:13.762234 18917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:50:13.770202 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:13.763609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58837 (local address 127.24.215.190:40535)
0810 14:50:13.764365 (+ 756us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:13.764376 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:13.764415 (+ 39us) server_negotiation.cc:407] Connection header received
0810 14:50:13.766689 (+ 2274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:13.766711 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:13.767164 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:13.767570 (+ 406us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:13.768672 (+ 1102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:13.768678 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:13.768690 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:50:13.768708 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:13.768766 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:50:13.768784 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:13.769040 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:13.769190 (+ 150us) server_negotiation.cc:299] Negotiation successful
0810 14:50:13.769359 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":324,"threads_started":1}
I20260810 14:50:13.773801 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:50:13.775130 18917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c: Bootstrap starting.
I20260810 14:50:13.778446 18917 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c: Neither blocks nor log segments found. Creating new log.
I20260810 14:50:13.779763 18917 log.cc:824] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c: Log is configured to *not* fsync() on all Append() calls
I20260810 14:50:13.780889 18920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:13.778333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55603 (local address 127.24.215.190:40535)
0810 14:50:13.778534 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:13.778541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:13.778931 (+ 390us) server_negotiation.cc:407] Connection header received
0810 14:50:13.779152 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:13.779158 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:13.779215 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:13.779313 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:13.780243 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:13.780249 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:13.780253 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:50:13.780278 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:13.780314 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:50:13.780328 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:13.780457 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:13.780567 (+ 110us) server_negotiation.cc:299] Negotiation successful
0810 14:50:13.780624 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260810 14:50:13.782910 18917 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c: No bootstrap required, opened a new log
I20260810 14:50:13.790385 18917 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed09e9e9f5434756b6ef43a1df75764c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40535 } }
I20260810 14:50:13.790755 18917 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:50:13.790875 18917 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed09e9e9f5434756b6ef43a1df75764c, State: Initialized, Role: FOLLOWER
I20260810 14:50:13.791497 18917 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed09e9e9f5434756b6ef43a1df75764c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40535 } }
I20260810 14:50:13.791728 18917 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:50:13.791883 18917 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:50:13.792063 18917 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:50:13.794224 18917 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed09e9e9f5434756b6ef43a1df75764c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40535 } }
I20260810 14:50:13.794756 18917 leader_election.cc:302] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed09e9e9f5434756b6ef43a1df75764c; no voters:
I20260810 14:50:13.795399 18917 leader_election.cc:288] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:50:13.795506 18928 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:50:13.796231 18928 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [term 1 LEADER]: Becoming Leader. State: Replica: ed09e9e9f5434756b6ef43a1df75764c, State: Running, Role: LEADER
I20260810 14:50:13.796794 18928 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed09e9e9f5434756b6ef43a1df75764c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40535 } }
I20260810 14:50:13.797722 18917 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:50:13.800009 18930 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed09e9e9f5434756b6ef43a1df75764c. Latest consensus state: current_term: 1 leader_uuid: "ed09e9e9f5434756b6ef43a1df75764c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed09e9e9f5434756b6ef43a1df75764c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40535 } } }
I20260810 14:50:13.800103 18929 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed09e9e9f5434756b6ef43a1df75764c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed09e9e9f5434756b6ef43a1df75764c" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 40535 } } }
I20260810 14:50:13.800407 18930 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:13.800513 18929 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:13.802832 18935 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:50:13.808570 18935 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:50:13.818255 18935 catalog_manager.cc:1347] Generated new cluster ID: 2a800340c28c47fcb211e17c49519477
I20260810 14:50:13.818396 18935 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:50:13.832927 18935 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:50:13.833508 18935 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:50:13.843636 18935 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed09e9e9f5434756b6ef43a1df75764c: Generated new TSK 0
I20260810 14:50:13.844731 18935 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:50:13.857080 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 18839
W20260810 14:50:13.858283 18910 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:58837
W20260810 14:50:13.859033 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 18839: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:50:13.859280 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 18839
2026-08-10T14:50:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2067 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:50:13 dist-test-slave-vkjp krb5kdc[18950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:50:13 dist-test-slave-vkjp krb5kdc[18950](info): set up 2 sockets
Aug 10 14:50:13 dist-test-slave-vkjp krb5kdc[18950](info): commencing operation
krb5kdc: starting...
W20260810 14:50:15.087759 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.186s user 0.005s sys 0.038s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:50:15 dist-test-slave-vkjp krb5kdc[18950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:50:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:50:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:50:15.410195 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:39311
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:38689
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:39311
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:50:15.755185 18964 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:50:15.755535 18964 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:50:15.755716 18964 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:50:15.762913 18964 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:50:15.763062 18964 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:50:15.763159 18964 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:50:15.763263 18964 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:50:15.763329 18964 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:50:15.773094 18964 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:38689
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:39311
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:39311
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:50:15.774438 18964 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:50:15.775892 18964 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:50:15.787770 18970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:15.787904 18971 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:15.787794 18973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:50:15.788915 18964 server_base.cc:1034] running on GCE node
I20260810 14:50:15.789709 18964 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:50:15.791179 18964 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:50:15.792415 18964 hybrid_clock.cc:648] HybridClock initialized: now 1786373415792359 us; error 62 us; skew 500 ppm
I20260810 14:50:15.792779 18964 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:50:15 dist-test-slave-vkjp krb5kdc[18950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373415, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:50:15.800817 18964 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:50:15.802484 18964 webserver.cc:460] Webserver started at http://127.24.215.190:39189/ using document root <none> and password file <none>
I20260810 14:50:15.803314 18964 fs_manager.cc:359] Metadata directory not provided
I20260810 14:50:15.803493 18964 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:50:15.803820 18964 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:50:15.806874 18964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "153b55c7656445b9b5408d823bc3dee7"
format_stamp: "Formatted at 2026-08-10 14:50:15 on dist-test-slave-vkjp"
server_key: "cd50325baa8d154a1e8c75e385cbfa4f"
server_key_iv: "51906f7933a432f14e36a930d9c71bad"
server_key_version: "encryptionkey@0"
I20260810 14:50:15.807726 18964 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "153b55c7656445b9b5408d823bc3dee7"
format_stamp: "Formatted at 2026-08-10 14:50:15 on dist-test-slave-vkjp"
server_key: "cd50325baa8d154a1e8c75e385cbfa4f"
server_key_iv: "51906f7933a432f14e36a930d9c71bad"
server_key_version: "encryptionkey@0"
I20260810 14:50:15.812911 18964 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:50:15.816324 18981 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:50:15.817711 18964 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:50:15.817912 18964 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "153b55c7656445b9b5408d823bc3dee7"
format_stamp: "Formatted at 2026-08-10 14:50:15 on dist-test-slave-vkjp"
server_key: "cd50325baa8d154a1e8c75e385cbfa4f"
server_key_iv: "51906f7933a432f14e36a930d9c71bad"
server_key_version: "encryptionkey@0"
I20260810 14:50:15.818122 18964 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:50:15.831661 18964 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:50:15.835587 18964 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:50:15.835920 18964 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:50:15.855381 18964 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:39311
I20260810 14:50:15.855425 19039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:39311 every 8 connection(s)
I20260810 14:50:15.856804 18964 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:50:15.860563 19040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:50:15.861126 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 18964
I20260810 14:50:15.861306 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:50:15.861770 25438 external_mini_cluster.cc:1428] Setting key e77a187180a73f6034a65fc9afe1d065
I20260810 14:50:15.874564 19040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7: Bootstrap starting.
Aug 10 14:50:15 dist-test-slave-vkjp krb5kdc[18950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373415, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:50:15.878324 19040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7: Neither blocks nor log segments found. Creating new log.
I20260810 14:50:15.879505 19040 log.cc:824] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7: Log is configured to *not* fsync() on all Append() calls
I20260810 14:50:15.882809 19040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7: No bootstrap required, opened a new log
I20260810 14:50:15.886713 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:15.863533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46995 (local address 127.24.215.190:39311)
0810 14:50:15.864352 (+ 819us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:15.864365 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:15.864408 (+ 43us) server_negotiation.cc:407] Connection header received
0810 14:50:15.866606 (+ 2198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:15.866634 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:15.867033 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:15.867452 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:15.869326 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:15.870607 (+ 1281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:15.872507 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:15.873148 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:15.873280 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:15.878072 (+ 4792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:15.878080 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:15.878095 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:15.878137 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:15.881967 (+ 3830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:15.882669 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:15.882677 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:15.882685 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:15.882950 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:15.883407 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:15.883412 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:15.883416 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:15.884148 (+ 732us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:15.884447 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:15.884827 (+ 380us) server_negotiation.cc:299] Negotiation successful
0810 14:50:15.885316 (+ 489us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":384,"threads_started":1}
I20260810 14:50:15.890686 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:50:15.892648 19040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153b55c7656445b9b5408d823bc3dee7" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39311 } }
I20260810 14:50:15.893074 19040 raft_consensus.cc:374] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:50:15.893194 19040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 153b55c7656445b9b5408d823bc3dee7, State: Initialized, Role: FOLLOWER
I20260810 14:50:15.893838 19040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153b55c7656445b9b5408d823bc3dee7" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39311 } }
I20260810 14:50:15.894168 19040 raft_consensus.cc:388] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:50:15.894333 19040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:50:15.894553 19040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:50:15.897593 19040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153b55c7656445b9b5408d823bc3dee7" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39311 } }
I20260810 14:50:15.898089 19040 leader_election.cc:302] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 153b55c7656445b9b5408d823bc3dee7; no voters:
I20260810 14:50:15.898828 19040 leader_election.cc:288] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:50:15.899016 19053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:50:15.899701 19053 raft_consensus.cc:686] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [term 1 LEADER]: Becoming Leader. State: Replica: 153b55c7656445b9b5408d823bc3dee7, State: Running, Role: LEADER
I20260810 14:50:15.900413 19053 consensus_queue.cc:237] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153b55c7656445b9b5408d823bc3dee7" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39311 } }
I20260810 14:50:15.902761 19053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 153b55c7656445b9b5408d823bc3dee7. Latest consensus state: current_term: 1 leader_uuid: "153b55c7656445b9b5408d823bc3dee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153b55c7656445b9b5408d823bc3dee7" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39311 } } }
I20260810 14:50:15.903229 19053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:15.903645 19055 sys_catalog.cc:455] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "153b55c7656445b9b5408d823bc3dee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "153b55c7656445b9b5408d823bc3dee7" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39311 } } }
I20260810 14:50:15.904052 19055 sys_catalog.cc:458] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:15.907593 19040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:50:15.912099 19057 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:50:15.914361 19044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:15.896060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43919 (local address 127.24.215.190:39311)
0810 14:50:15.896252 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:15.896260 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:15.896737 (+ 477us) server_negotiation.cc:407] Connection header received
0810 14:50:15.896883 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:15.896889 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:15.896969 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:15.897070 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:15.898604 (+ 1534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:15.899797 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:15.901340 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:15.901790 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:15.901923 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:15.906134 (+ 4211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:15.906144 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:15.906152 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:15.906204 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:15.911307 (+ 5103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:15.912106 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:15.912110 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:15.912112 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:15.912284 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:15.912703 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:15.912708 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:15.912711 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:15.913087 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:15.913361 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:15.913719 (+ 358us) server_negotiation.cc:299] Negotiation successful
0810 14:50:15.913960 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260810 14:50:15.917140 19057 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:50:15.928562 19057 catalog_manager.cc:1347] Generated new cluster ID: 4e27c22406474988a7723ecd0b945d64
I20260810 14:50:15.928711 19057 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:50:15.957281 19057 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:50:15.958962 19057 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:50:15.965570 19057 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 153b55c7656445b9b5408d823bc3dee7: Generated new TSK 0
I20260810 14:50:15.966390 19057 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:50:15.985093 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 18964
I20260810 14:50:15.986348 19035 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:50:16.079792 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 18964
2026-08-10T14:50:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2217 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:50:16 dist-test-slave-vkjp krb5kdc[19077](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:50:16 dist-test-slave-vkjp krb5kdc[19077](info): set up 2 sockets
Aug 10 14:50:16 dist-test-slave-vkjp krb5kdc[19077](info): commencing operation
krb5kdc: starting...
W20260810 14:50:17.277253 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.002s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:50:17 dist-test-slave-vkjp krb5kdc[19077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373417, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:50:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:50:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:50:17.584476 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:39671
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:44931
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:39671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:50:17.929854 19093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:50:17.930153 19093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:50:17.930320 19093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:50:17.937584 19093 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:50:17.937700 19093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:50:17.937803 19093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:50:17.937907 19093 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:50:17.938001 19093 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:50:17.947872 19093 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:44931
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:39671
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:39671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:50:17.949254 19093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:50:17.950683 19093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:50:17.963016 19099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:17.963496 19098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:17.963531 19101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:50:17.964167 19093 server_base.cc:1034] running on GCE node
I20260810 14:50:17.965035 19093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:50:17.966533 19093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:50:17.967765 19093 hybrid_clock.cc:648] HybridClock initialized: now 1786373417967719 us; error 56 us; skew 500 ppm
I20260810 14:50:17.968148 19093 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:50:17 dist-test-slave-vkjp krb5kdc[19077](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373417, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:50:17.971313 19093 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:50:17.972813 19093 webserver.cc:460] Webserver started at http://127.24.215.190:33007/ using document root <none> and password file <none>
I20260810 14:50:17.973524 19093 fs_manager.cc:359] Metadata directory not provided
I20260810 14:50:17.973681 19093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:50:17.974004 19093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:50:17.977221 19093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "11f7d5537d934c6ebe51cda7f623dd86"
format_stamp: "Formatted at 2026-08-10 14:50:17 on dist-test-slave-vkjp"
server_key: "5b09cdb3782b4265d1b0b85d7f714f3f"
server_key_iv: "d4edfc28db16c8350bae617223ca5076"
server_key_version: "encryptionkey@0"
I20260810 14:50:17.978085 19093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "11f7d5537d934c6ebe51cda7f623dd86"
format_stamp: "Formatted at 2026-08-10 14:50:17 on dist-test-slave-vkjp"
server_key: "5b09cdb3782b4265d1b0b85d7f714f3f"
server_key_iv: "d4edfc28db16c8350bae617223ca5076"
server_key_version: "encryptionkey@0"
I20260810 14:50:17.983780 19093 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 14:50:17.987528 19111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:50:17.988871 19093 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260810 14:50:17.989089 19093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "11f7d5537d934c6ebe51cda7f623dd86"
format_stamp: "Formatted at 2026-08-10 14:50:17 on dist-test-slave-vkjp"
server_key: "5b09cdb3782b4265d1b0b85d7f714f3f"
server_key_iv: "d4edfc28db16c8350bae617223ca5076"
server_key_version: "encryptionkey@0"
I20260810 14:50:17.989312 19093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:50:18.001899 19093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:50:18.006114 19093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:50:18.006422 19093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:50:18.025480 19093 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:39671
I20260810 14:50:18.025499 19167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:39671 every 8 connection(s)
I20260810 14:50:18.026952 19093 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:50:18.027385 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 19093
I20260810 14:50:18.027582 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:50:18.028249 25438 external_mini_cluster.cc:1428] Setting key 7123e7995201684ffb9a9277555b6515
I20260810 14:50:18.031795 19168 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:50:18.042668 19171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:18.030210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53607 (local address 127.24.215.190:39671)
0810 14:50:18.031003 (+ 793us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:18.031019 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:18.031069 (+ 50us) server_negotiation.cc:407] Connection header received
0810 14:50:18.033489 (+ 2420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:18.033517 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:18.033980 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:18.034396 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:18.034947 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:18.036333 (+ 1386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:18.038219 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:18.038880 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:18.039011 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:18.040343 (+ 1332us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:18.040351 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:18.040364 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:50:18.040901 (+ 537us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:18.040991 (+ 90us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:50:18.041031 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:18.041121 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:18.041371 (+ 250us) server_negotiation.cc:299] Negotiation successful
0810 14:50:18.041657 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":338,"threads_started":1}
I20260810 14:50:18.044728 19168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86: Bootstrap starting.
I20260810 14:50:18.046317 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:50:18.048060 19168 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86: Neither blocks nor log segments found. Creating new log.
I20260810 14:50:18.049172 19168 log.cc:824] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86: Log is configured to *not* fsync() on all Append() calls
I20260810 14:50:18.052500 19168 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86: No bootstrap required, opened a new log
I20260810 14:50:18.057170 19171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:18.050996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52035 (local address 127.24.215.190:39671)
0810 14:50:18.051208 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:18.051215 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:18.051598 (+ 383us) server_negotiation.cc:407] Connection header received
0810 14:50:18.051815 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:18.051825 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:18.051886 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:18.052002 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:18.052514 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:18.053275 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:18.054610 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:18.055155 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:18.055327 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:18.056404 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:18.056413 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:18.056416 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:50:18.056433 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:18.056474 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:50:18.056493 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:18.056602 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:18.056753 (+ 151us) server_negotiation.cc:299] Negotiation successful
0810 14:50:18.056852 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260810 14:50:18.060788 19168 raft_consensus.cc:348] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f7d5537d934c6ebe51cda7f623dd86" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39671 } }
I20260810 14:50:18.061239 19168 raft_consensus.cc:374] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:50:18.061355 19168 raft_consensus.cc:729] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 11f7d5537d934c6ebe51cda7f623dd86, State: Initialized, Role: FOLLOWER
I20260810 14:50:18.061969 19168 consensus_queue.cc:260] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f7d5537d934c6ebe51cda7f623dd86" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39671 } }
I20260810 14:50:18.062245 19168 raft_consensus.cc:388] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:50:18.062415 19168 raft_consensus.cc:482] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:50:18.062614 19168 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:50:18.065109 19168 raft_consensus.cc:504] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f7d5537d934c6ebe51cda7f623dd86" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39671 } }
I20260810 14:50:18.065655 19168 leader_election.cc:302] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 11f7d5537d934c6ebe51cda7f623dd86; no voters:
I20260810 14:50:18.066262 19168 leader_election.cc:288] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:50:18.066390 19178 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:50:18.067787 19178 raft_consensus.cc:686] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [term 1 LEADER]: Becoming Leader. State: Replica: 11f7d5537d934c6ebe51cda7f623dd86, State: Running, Role: LEADER
I20260810 14:50:18.068293 19178 consensus_queue.cc:237] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f7d5537d934c6ebe51cda7f623dd86" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39671 } }
I20260810 14:50:18.068516 19168 sys_catalog.cc:564] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:50:18.071149 19180 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 11f7d5537d934c6ebe51cda7f623dd86. Latest consensus state: current_term: 1 leader_uuid: "11f7d5537d934c6ebe51cda7f623dd86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f7d5537d934c6ebe51cda7f623dd86" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39671 } } }
I20260810 14:50:18.071578 19180 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:18.071456 19179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "11f7d5537d934c6ebe51cda7f623dd86" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "11f7d5537d934c6ebe51cda7f623dd86" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 39671 } } }
I20260810 14:50:18.071713 19179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:18.072388 19184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:50:18.078759 19184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:50:18.088764 19184 catalog_manager.cc:1347] Generated new cluster ID: 4a2b087577594e43be30fc05a704f664
I20260810 14:50:18.088910 19184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:50:18.101406 19184 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:50:18.102824 19184 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:50:18.110826 19184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 11f7d5537d934c6ebe51cda7f623dd86: Generated new TSK 0
I20260810 14:50:18.111740 19184 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:50:18.122488 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 19093
W20260810 14:50:18.123800 19162 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:53607
W20260810 14:50:18.124534 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 19093: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:50:18.124696 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 19093
2026-08-10T14:50:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2047 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:50:18 dist-test-slave-vkjp krb5kdc[19201](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:50:18 dist-test-slave-vkjp krb5kdc[19201](info): set up 2 sockets
Aug 10 14:50:18 dist-test-slave-vkjp krb5kdc[19201](info): commencing operation
krb5kdc: starting...
W20260810 14:50:19.332657 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.002s sys 0.040s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:50:19 dist-test-slave-vkjp krb5kdc[19201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373419, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:50:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:50:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:50:19.646672 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:44515
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:37603
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:44515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:50:19.993626 19216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:50:19.993996 19216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:50:19.994172 19216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:50:20.001586 19216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:50:20.001766 19216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:50:20.001874 19216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:50:20.001998 19216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:50:20.002084 19216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:50:20.012331 19216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:37603
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:44515
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:44515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:50:20.013734 19216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:50:20.015146 19216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:50:20.031754 19222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:20.032452 19223 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:20.032452 19225 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:50:20.033566 19216 server_base.cc:1034] running on GCE node
I20260810 14:50:20.034386 19216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:50:20.035938 19216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:50:20.037199 19216 hybrid_clock.cc:648] HybridClock initialized: now 1786373420037153 us; error 38 us; skew 500 ppm
I20260810 14:50:20.037575 19216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:50:20 dist-test-slave-vkjp krb5kdc[19201](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373420, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:50:20.040742 19216 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:50:20.042279 19216 webserver.cc:460] Webserver started at http://127.24.215.190:40019/ using document root <none> and password file <none>
I20260810 14:50:20.043005 19216 fs_manager.cc:359] Metadata directory not provided
I20260810 14:50:20.043190 19216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:50:20.043524 19216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:50:20.046567 19216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "ee54573c088c402f876ae6fabd906ac2"
format_stamp: "Formatted at 2026-08-10 14:50:20 on dist-test-slave-vkjp"
server_key: "72610e33316e63e0b0eea09387528dbd"
server_key_iv: "355b4d47a1d34ac20e78a3db90a1d3ee"
server_key_version: "encryptionkey@0"
I20260810 14:50:20.047420 19216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "ee54573c088c402f876ae6fabd906ac2"
format_stamp: "Formatted at 2026-08-10 14:50:20 on dist-test-slave-vkjp"
server_key: "72610e33316e63e0b0eea09387528dbd"
server_key_iv: "355b4d47a1d34ac20e78a3db90a1d3ee"
server_key_version: "encryptionkey@0"
I20260810 14:50:20.052721 19216 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260810 14:50:20.056402 19232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:50:20.057730 19216 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260810 14:50:20.057915 19216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "ee54573c088c402f876ae6fabd906ac2"
format_stamp: "Formatted at 2026-08-10 14:50:20 on dist-test-slave-vkjp"
server_key: "72610e33316e63e0b0eea09387528dbd"
server_key_iv: "355b4d47a1d34ac20e78a3db90a1d3ee"
server_key_version: "encryptionkey@0"
I20260810 14:50:20.058125 19216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:50:20.064060 19216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:50:20.068226 19216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:50:20.068531 19216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:50:20.086974 19216 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:44515
I20260810 14:50:20.087004 19286 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:44515 every 8 connection(s)
I20260810 14:50:20.088378 19216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:50:20.092079 19287 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:50:20.097008 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 19216
I20260810 14:50:20.097199 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:50:20.097622 25438 external_mini_cluster.cc:1428] Setting key 584b24191b4449ca9ac48ab9ad78a797
I20260810 14:50:20.102967 19287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2: Bootstrap starting.
I20260810 14:50:20.105448 19291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:20.099412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59931 (local address 127.24.215.190:44515)
0810 14:50:20.100270 (+ 858us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:20.100282 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:20.100323 (+ 41us) server_negotiation.cc:407] Connection header received
0810 14:50:20.102054 (+ 1731us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:20.102076 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:20.102388 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:20.102701 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:20.103916 (+ 1215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:20.103923 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:20.103935 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:50:20.103949 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:20.104013 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:50:20.104033 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:20.104197 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:20.104334 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 14:50:20.104531 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":629,"thread_start_us":407,"threads_started":1}
I20260810 14:50:20.106532 19287 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2: Neither blocks nor log segments found. Creating new log.
I20260810 14:50:20.107677 19287 log.cc:824] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2: Log is configured to *not* fsync() on all Append() calls
I20260810 14:50:20.109119 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:50:20.110628 19287 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2: No bootstrap required, opened a new log
I20260810 14:50:20.116407 19291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:20.113604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53373 (local address 127.24.215.190:44515)
0810 14:50:20.113839 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:20.113845 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:20.114222 (+ 377us) server_negotiation.cc:407] Connection header received
0810 14:50:20.114372 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:20.114375 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:20.114419 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:20.114511 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:20.115677 (+ 1166us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:20.115686 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:20.115691 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0810 14:50:20.115713 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:20.115790 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0810 14:50:20.115818 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:20.115916 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:20.116053 (+ 137us) server_negotiation.cc:299] Negotiation successful
0810 14:50:20.116100 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260810 14:50:20.119298 19287 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee54573c088c402f876ae6fabd906ac2" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44515 } }
I20260810 14:50:20.119663 19287 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260810 14:50:20.119787 19287 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee54573c088c402f876ae6fabd906ac2, State: Initialized, Role: FOLLOWER
I20260810 14:50:20.120322 19287 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee54573c088c402f876ae6fabd906ac2" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44515 } }
I20260810 14:50:20.120571 19287 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260810 14:50:20.120721 19287 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260810 14:50:20.120898 19287 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:50:20.123075 19287 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee54573c088c402f876ae6fabd906ac2" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44515 } }
I20260810 14:50:20.123637 19287 leader_election.cc:302] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee54573c088c402f876ae6fabd906ac2; no voters:
I20260810 14:50:20.124222 19287 leader_election.cc:288] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260810 14:50:20.124347 19300 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:50:20.125115 19300 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [term 1 LEADER]: Becoming Leader. State: Replica: ee54573c088c402f876ae6fabd906ac2, State: Running, Role: LEADER
I20260810 14:50:20.125702 19300 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee54573c088c402f876ae6fabd906ac2" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44515 } }
I20260810 14:50:20.126415 19287 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:50:20.128499 19302 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee54573c088c402f876ae6fabd906ac2. Latest consensus state: current_term: 1 leader_uuid: "ee54573c088c402f876ae6fabd906ac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee54573c088c402f876ae6fabd906ac2" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44515 } } }
I20260810 14:50:20.128630 19301 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee54573c088c402f876ae6fabd906ac2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee54573c088c402f876ae6fabd906ac2" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44515 } } }
I20260810 14:50:20.129025 19302 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:20.129022 19301 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:20.131425 19309 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:50:20.136631 19309 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:50:20.146106 19309 catalog_manager.cc:1347] Generated new cluster ID: c5666aa71b874628b043f6511b61d150
I20260810 14:50:20.146234 19309 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:50:20.159158 19309 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:50:20.159644 19309 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:50:20.168131 19309 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee54573c088c402f876ae6fabd906ac2: Generated new TSK 0
I20260810 14:50:20.168959 19309 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260810 14:50:20.179984 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 19216
W20260810 14:50:20.181149 19282 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:59931
W20260810 14:50:20.181838 25438 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 19216: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260810 14:50:20.182003 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 19216
2026-08-10T14:50:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2056 ms)
[----------] 18 tests from AuthTokenIssuingTest (38200 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:50:20 dist-test-slave-vkjp krb5kdc[19324](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:50:20 dist-test-slave-vkjp krb5kdc[19324](info): set up 2 sockets
Aug 10 14:50:20 dist-test-slave-vkjp krb5kdc[19324](info): commencing operation
krb5kdc: starting...
W20260810 14:50:21.419935 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.003s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:50:21 dist-test-slave-vkjp krb5kdc[19324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:50:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:50:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:50:21.739496 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:34261
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:39843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:34261,127.24.215.189:41363,127.24.215.188:38169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:50:22.085958 19337 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:50:22.086272 19337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:50:22.086351 19337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:50:22.086433 19337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:50:22.093814 19337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:50:22.093902 19337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:50:22.093961 19337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:50:22.094012 19337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:50:22.094072 19337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:50:22.103998 19337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:39843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:34261,127.24.215.189:41363,127.24.215.188:38169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:34261
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:50:22.105266 19337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:50:22.106575 19337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:50:22.119024 19344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:22.119545 19343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:22.119521 19347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:50:22.120427 19337 server_base.cc:1034] running on GCE node
I20260810 14:50:22.121182 19337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:50:22.122531 19337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:50:22.123761 19337 hybrid_clock.cc:648] HybridClock initialized: now 1786373422123692 us; error 34 us; skew 500 ppm
I20260810 14:50:22.124080 19337 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:50:22 dist-test-slave-vkjp krb5kdc[19324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:50:22.130764 19337 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:50:22.132405 19337 webserver.cc:460] Webserver started at http://127.24.215.190:33947/ using document root <none> and password file <none>
I20260810 14:50:22.133133 19337 fs_manager.cc:359] Metadata directory not provided
I20260810 14:50:22.133261 19337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:50:22.133613 19337 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:50:22.136786 19337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "eee1732542c54b5498b72d1298461d10"
format_stamp: "Formatted at 2026-08-10 14:50:22 on dist-test-slave-vkjp"
server_key: "25dce07c0fae340f5d4b43b8ccf10371"
server_key_iv: "92ddc11e545d22daa9b3a01d78c001cf"
server_key_version: "encryptionkey@0"
I20260810 14:50:22.137639 19337 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "eee1732542c54b5498b72d1298461d10"
format_stamp: "Formatted at 2026-08-10 14:50:22 on dist-test-slave-vkjp"
server_key: "25dce07c0fae340f5d4b43b8ccf10371"
server_key_iv: "92ddc11e545d22daa9b3a01d78c001cf"
server_key_version: "encryptionkey@0"
I20260810 14:50:22.142982 19337 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 14:50:22.146651 19355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:50:22.148142 19337 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:50:22.148331 19337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "eee1732542c54b5498b72d1298461d10"
format_stamp: "Formatted at 2026-08-10 14:50:22 on dist-test-slave-vkjp"
server_key: "25dce07c0fae340f5d4b43b8ccf10371"
server_key_iv: "92ddc11e545d22daa9b3a01d78c001cf"
server_key_version: "encryptionkey@0"
I20260810 14:50:22.148583 19337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:50:22.170369 19337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:50:22.174357 19337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:50:22.174649 19337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:50:22.193401 19337 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:34261
I20260810 14:50:22.193432 19413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:34261 every 8 connection(s)
I20260810 14:50:22.194780 19337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:50:22.198422 19414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:50:22.200824 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 19337
I20260810 14:50:22.201076 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:50:22.201727 25438 external_mini_cluster.cc:1428] Setting key 0ff6ca5625841e2577616992e6db295b
I20260810 14:50:22.205090 19414 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:50:22 dist-test-slave-vkjp krb5kdc[19324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:50:22.227308 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.208375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34261 (local address 127.0.0.1:53102)
0810 14:50:22.209147 (+ 772us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.209191 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:22.211328 (+ 2137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:22.212137 (+ 809us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:22.212144 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:22.212832 (+ 688us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:22.213163 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:22.213180 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:22.214743 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:22.214749 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:22.215873 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:22.215885 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:22.216804 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:22.216810 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:22.217007 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:22.217718 (+ 711us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:22.217741 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:22.220011 (+ 2270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:22.223215 (+ 3204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:22.223226 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:22.223238 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:22.223619 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:22.224021 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:22.224026 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:22.224029 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:22.224182 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:22.224905 (+ 723us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:22.224913 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:22.225119 (+ 206us) client_negotiation.cc:769] Sending connection context
0810 14:50:22.225357 (+ 238us) client_negotiation.cc:240] Negotiation successful
0810 14:50:22.225968 (+ 611us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":150,"thread_start_us":306,"threads_started":1}
I20260810 14:50:22.227327 19417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.209662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53102 (local address 127.24.215.190:34261)
0810 14:50:22.210439 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:22.210456 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:22.210504 (+ 48us) server_negotiation.cc:407] Connection header received
0810 14:50:22.211691 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:22.211696 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:22.211912 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:22.212116 (+ 204us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:22.213381 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:22.214587 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:22.216022 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:22.216570 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:22.216769 (+ 199us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:22.220235 (+ 3466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:22.220242 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:22.220254 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:22.220281 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:22.223031 (+ 2750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:22.223752 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:22.223766 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:22.223769 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:22.223905 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:22.224295 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:22.224306 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:22.224308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:22.224767 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:22.224970 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:22.225478 (+ 508us) server_negotiation.cc:299] Negotiation successful
0810 14:50:22.225850 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":553,"thread_start_us":373,"threads_started":1}
I20260810 14:50:22.230654 19414 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.215.189@KRBTEST.COM; defaulting to no policy
I20260810 14:50:22.231678 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.231288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:41363 (local address 127.0.0.1:43032)
0810 14:50:22.231467 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.231566 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260810 14:50:22.231910 19358 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.215.189:41363: connect: Connection refused (error 111)
I20260810 14:50:22.233364 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.233011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:41363 (local address 127.0.0.1:43046)
0810 14:50:22.233170 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.233231 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 14:50:22.233613 19414 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.189:41363: Network error: Client connection negotiation failed: client connection to 127.24.215.189:41363: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.189.keytab.
Entry for principal kudu/127.24.215.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.189.keytab.
I20260810 14:50:22.270148 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.189:41363
--webserver_interface=127.24.215.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.189.keytab
--principal=kudu/127.24.215.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:39843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:34261,127.24.215.189:41363,127.24.215.188:38169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:50:22.282965 19414 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } attempt: 1
I20260810 14:50:22.284238 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.283791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:41363 (local address 127.0.0.1:43050)
0810 14:50:22.283997 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.284112 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 14:50:22.285220 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.284862 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:41363 (local address 127.0.0.1:43058)
0810 14:50:22.285024 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.285092 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 14:50:22.285405 19414 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.189:41363: Network error: Client connection negotiation failed: client connection to 127.24.215.189:41363: connect: Connection refused (error 111)
I20260810 14:50:22.353632 19414 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } attempt: 2
I20260810 14:50:22.354858 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.354296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:41363 (local address 127.0.0.1:43068)
0810 14:50:22.354554 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.354689 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260810 14:50:22.355903 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.355530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:41363 (local address 127.0.0.1:43070)
0810 14:50:22.355694 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.355773 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 14:50:22.356083 19414 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.189:41363: Network error: Client connection negotiation failed: client connection to 127.24.215.189:41363: connect: Connection refused (error 111)
I20260810 14:50:22.447337 19414 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } attempt: 3
I20260810 14:50:22.448447 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.448013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:41363 (local address 127.0.0.1:43076)
0810 14:50:22.448221 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.448317 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 14:50:22.449424 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.449075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:41363 (local address 127.0.0.1:43078)
0810 14:50:22.449217 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.449290 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 14:50:22.449601 19414 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.189:41363: Network error: Client connection negotiation failed: client connection to 127.24.215.189:41363: connect: Connection refused (error 111)
I20260810 14:50:22.592931 19414 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } attempt: 4
I20260810 14:50:22.594131 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.593598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:41363 (local address 127.0.0.1:43084)
0810 14:50:22.593907 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.594001 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260810 14:50:22.595120 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.594739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:41363 (local address 127.0.0.1:43098)
0810 14:50:22.594883 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.594963 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:41363: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260810 14:50:22.595304 19414 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.189:41363: Network error: Client connection negotiation failed: client connection to 127.24.215.189:41363: connect: Connection refused (error 111)
W20260810 14:50:22.627651 19421 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:50:22.628042 19421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:50:22.628173 19421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:50:22.628321 19421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:50:22.635617 19421 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:50:22.635751 19421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:50:22.635864 19421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:50:22.635954 19421 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:50:22.636047 19421 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:50:22.646236 19421 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:39843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:34261,127.24.215.189:41363,127.24.215.188:38169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.189.keytab
--principal=kudu/127.24.215.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.189:41363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.215.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:50:22.647635 19421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:50:22.648963 19421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:50:22.660712 19426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:22.660787 19427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:22.660784 19429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:50:22.661763 19421 server_base.cc:1034] running on GCE node
I20260810 14:50:22.662524 19421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:50:22.664021 19421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:50:22.665266 19421 hybrid_clock.cc:648] HybridClock initialized: now 1786373422665204 us; error 34 us; skew 500 ppm
I20260810 14:50:22.665607 19421 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:50:22 dist-test-slave-vkjp krb5kdc[19324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:50:22.668866 19421 init.cc:377] Logged in from keytab as kudu/127.24.215.189@KRBTEST.COM (short username kudu)
I20260810 14:50:22.670254 19421 webserver.cc:460] Webserver started at http://127.24.215.189:42071/ using document root <none> and password file <none>
I20260810 14:50:22.670958 19421 fs_manager.cc:359] Metadata directory not provided
I20260810 14:50:22.671132 19421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:50:22.671423 19421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:50:22.674257 19421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/data/instance:
uuid: "bfed2fbc273f4c47952d135432e1ee8a"
format_stamp: "Formatted at 2026-08-10 14:50:22 on dist-test-slave-vkjp"
server_key: "520e7951f13a049b412e3fa00aa67a70"
server_key_iv: "6160a75f16e23943c3ba0970a5d79423"
server_key_version: "encryptionkey@0"
I20260810 14:50:22.674983 19421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/wal/instance:
uuid: "bfed2fbc273f4c47952d135432e1ee8a"
format_stamp: "Formatted at 2026-08-10 14:50:22 on dist-test-slave-vkjp"
server_key: "520e7951f13a049b412e3fa00aa67a70"
server_key_iv: "6160a75f16e23943c3ba0970a5d79423"
server_key_version: "encryptionkey@0"
I20260810 14:50:22.680073 19421 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260810 14:50:22.683460 19436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:50:22.684782 19421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260810 14:50:22.684924 19421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/wal
uuid: "bfed2fbc273f4c47952d135432e1ee8a"
format_stamp: "Formatted at 2026-08-10 14:50:22 on dist-test-slave-vkjp"
server_key: "520e7951f13a049b412e3fa00aa67a70"
server_key_iv: "6160a75f16e23943c3ba0970a5d79423"
server_key_version: "encryptionkey@0"
I20260810 14:50:22.685098 19421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:50:22.707906 19421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:50:22.712473 19421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:50:22.712711 19421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:50:22.731127 19496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.189:41363 every 8 connection(s)
I20260810 14:50:22.731138 19421 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.189:41363
I20260810 14:50:22.732481 19421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/data/info.pb
I20260810 14:50:22.736227 19497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:50:22.742578 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 19421
I20260810 14:50:22.742784 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-1/wal/instance
I20260810 14:50:22.743366 25438 external_mini_cluster.cc:1428] Setting key 7824537bdb102eb16b04158a208c505a
I20260810 14:50:22.742764 19497 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:50:22 dist-test-slave-vkjp krb5kdc[19324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.189@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:50:22.765914 19499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.745951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53118 (local address 127.24.215.190:34261)
0810 14:50:22.746551 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:22.746560 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:22.747472 (+ 912us) server_negotiation.cc:407] Connection header received
0810 14:50:22.749321 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:22.749330 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:22.749419 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:22.749556 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:22.751033 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:22.752306 (+ 1273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:22.754055 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:22.754795 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:22.754939 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:22.758551 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:22.758561 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:22.758569 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:22.758620 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:22.762436 (+ 3816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:22.763208 (+ 772us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:22.763216 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:22.763222 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:22.763401 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:22.763839 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:22.763847 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:22.763852 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:22.764265 (+ 413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:22.764423 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:22.765150 (+ 727us) server_negotiation.cc:299] Negotiation successful
0810 14:50:22.765410 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":274,"threads_started":1}
I20260810 14:50:22.766585 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.746481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34261 (local address 127.0.0.1:53118)
0810 14:50:22.747292 (+ 811us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.747346 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:22.749054 (+ 1708us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:22.749754 (+ 700us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:22.749762 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:22.750272 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:22.750813 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:22.750830 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:22.752517 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:22.752521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:22.753903 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:22.753915 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:22.755021 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:22.755025 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:22.755289 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:22.755983 (+ 694us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:22.756017 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:22.758309 (+ 2292us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:22.762598 (+ 4289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:22.762606 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:22.762618 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:22.763056 (+ 438us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:22.763541 (+ 485us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:22.763544 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:22.763547 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:22.763715 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:22.764387 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:22.764393 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:22.764745 (+ 352us) client_negotiation.cc:769] Sending connection context
0810 14:50:22.765020 (+ 275us) client_negotiation.cc:240] Negotiation successful
0810 14:50:22.765435 (+ 415us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":530,"thread_start_us":331,"threads_started":1}
I20260810 14:50:22.768574 19497 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.215.188@KRBTEST.COM; defaulting to no policy
Aug 10 14:50:22 dist-test-slave-vkjp krb5kdc[19324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.189@KRBTEST.COM for kudu/127.24.215.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.188@KRBTEST.COM" created.
I20260810 14:50:22.784379 19502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.769592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43110 (local address 127.24.215.189:41363)
0810 14:50:22.770272 (+ 680us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:22.770278 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:22.770304 (+ 26us) server_negotiation.cc:407] Connection header received
0810 14:50:22.770346 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:22.770352 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:22.770543 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:22.770702 (+ 159us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:22.771537 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:22.772588 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:22.773728 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:22.774661 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:22.774791 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:22.778314 (+ 3523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:22.778326 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:22.778341 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:22.778374 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:22.781156 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:22.782081 (+ 925us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:22.782087 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:22.782089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:22.782203 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:22.782663 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:22.782666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:22.782668 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:22.783023 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:22.783284 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:22.783724 (+ 440us) server_negotiation.cc:299] Negotiation successful
0810 14:50:22.784077 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":141,"server-negotiator.queue_time_us":533,"thread_start_us":398,"threads_started":1}
I20260810 14:50:22.784461 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.769266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:41363 (local address 127.0.0.1:43110)
0810 14:50:22.769449 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.769465 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:22.769553 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:22.770691 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:22.770696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:22.771218 (+ 522us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:22.771350 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:22.771359 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:22.772731 (+ 1372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:22.772738 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:22.773594 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:22.773604 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:22.774996 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:22.775002 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:22.775209 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:22.775852 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:22.775868 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:22.778112 (+ 2244us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:22.781311 (+ 3199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:22.781318 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:22.781361 (+ 43us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:22.781940 (+ 579us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:22.782333 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:22.782340 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:22.782347 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:22.782551 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:22.783233 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:22.783241 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:22.783425 (+ 184us) client_negotiation.cc:769] Sending connection context
0810 14:50:22.783593 (+ 168us) client_negotiation.cc:240] Negotiation successful
0810 14:50:22.783955 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260810 14:50:22.786983 19497 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:50:22.788341 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.787839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54546)
0810 14:50:22.788093 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.788190 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 14:50:22.788625 19438 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
I20260810 14:50:22.790295 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.789910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54556)
0810 14:50:22.790119 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.790194 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260810 14:50:22.790530 19497 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:38169: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.188.keytab.
Entry for principal kudu/127.24.215.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.188.keytab.
I20260810 14:50:22.818454 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.188:38169
--webserver_interface=127.24.215.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.188.keytab
--principal=kudu/127.24.215.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:39843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:34261,127.24.215.189:41363,127.24.215.188:38169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:50:22.839831 19497 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } attempt: 1
I20260810 14:50:22.841115 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.840637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54572)
0810 14:50:22.840837 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.840945 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 14:50:22.842219 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.841845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54574)
0810 14:50:22.842026 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.842103 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260810 14:50:22.842432 19497 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:38169: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
I20260810 14:50:22.894580 19414 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } attempt: 5
Aug 10 14:50:22 dist-test-slave-vkjp krb5kdc[19324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for kudu/127.24.215.189@KRBTEST.COM
I20260810 14:50:22.909621 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.895440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:41363 (local address 127.0.0.1:43120)
0810 14:50:22.895681 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.895702 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:22.895855 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:22.896285 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:22.896289 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:22.896653 (+ 364us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:22.896828 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:22.896836 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:22.898322 (+ 1486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:22.898325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:22.899276 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:22.899298 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:22.900272 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:22.900277 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:22.900439 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:22.901116 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:22.901134 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:22.903254 (+ 2120us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:22.907201 (+ 3947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:22.907206 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:22.907208 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:22.907581 (+ 373us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:22.908045 (+ 464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:22.908048 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:22.908050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:22.908177 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:22.908894 (+ 717us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:22.908897 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:22.909030 (+ 133us) client_negotiation.cc:769] Sending connection context
0810 14:50:22.909140 (+ 110us) client_negotiation.cc:240] Negotiation successful
0810 14:50:22.909365 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 14:50:22.909974 19502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.895697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43120 (local address 127.24.215.189:41363)
0810 14:50:22.895944 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:22.895951 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:22.895975 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:50:22.896044 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:22.896049 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:22.896131 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:22.896249 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:22.897008 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:22.898187 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:22.899451 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:22.900129 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:22.900273 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:22.903472 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:22.903481 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:22.903487 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:22.903536 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:22.907001 (+ 3465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:22.907717 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:22.907725 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:22.907730 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:22.907919 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:22.908294 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:22.908301 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:22.908306 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:22.908756 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:22.908952 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:22.909323 (+ 371us) server_negotiation.cc:299] Negotiation successful
0810 14:50:22.909611 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260810 14:50:22.910629 19497 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } attempt: 2
I20260810 14:50:22.911158 19414 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:50:22.911597 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.911154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54576)
0810 14:50:22.911345 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.911442 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 14:50:22.912375 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.911954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54584)
0810 14:50:22.912115 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.912207 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260810 14:50:22.913056 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.912662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54598)
0810 14:50:22.912869 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.912946 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104}
W20260810 14:50:22.913290 19497 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:38169: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
I20260810 14:50:22.913596 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.913025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54600)
0810 14:50:22.913218 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.913503 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
W20260810 14:50:22.913800 19414 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:38169: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
I20260810 14:50:22.965052 19414 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } attempt: 1
I20260810 14:50:22.966363 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.965871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54604)
0810 14:50:22.966101 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.966195 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260810 14:50:22.967438 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:22.967057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54608)
0810 14:50:22.967250 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:22.967332 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
W20260810 14:50:22.967654 19414 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:38169: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
I20260810 14:50:23.004613 19497 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } attempt: 3
I20260810 14:50:23.005869 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.005414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54620)
0810 14:50:23.005616 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:23.005709 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260810 14:50:23.007033 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.006599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54626)
0810 14:50:23.006839 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:23.006916 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260810 14:50:23.007365 19497 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:38169: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
I20260810 14:50:23.035979 19414 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } attempt: 2
I20260810 14:50:23.037247 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.036708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54634)
0810 14:50:23.036983 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:23.037081 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260810 14:50:23.038231 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.037902 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54650)
0810 14:50:23.038063 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:23.038127 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260810 14:50:23.038444 19414 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:38169: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
I20260810 14:50:23.144655 19414 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } attempt: 3
I20260810 14:50:23.145839 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.145338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54664)
0810 14:50:23.145582 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:23.145669 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260810 14:50:23.146813 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.146454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54668)
0810 14:50:23.146640 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:23.146712 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260810 14:50:23.147029 19414 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:38169: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
I20260810 14:50:23.150574 19497 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } attempt: 4
I20260810 14:50:23.151618 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.151136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54672)
0810 14:50:23.151347 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:23.151474 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260810 14:50:23.152679 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.152309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54678)
0810 14:50:23.152470 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:23.152582 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 14:50:23.152925 19497 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:38169: Network error: Client connection negotiation failed: client connection to 127.24.215.188:38169: connect: Connection refused (error 111)
W20260810 14:50:23.174170 19505 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:50:23.174553 19505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:50:23.174679 19505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:50:23.174829 19505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:50:23.182267 19505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:50:23.182391 19505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:50:23.182502 19505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:50:23.182593 19505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:50:23.182693 19505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:50:23.192839 19505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:39843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:34261,127.24.215.189:41363,127.24.215.188:38169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.188.keytab
--principal=kudu/127.24.215.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.188:38169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.215.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:50:23.194187 19505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:50:23.195561 19505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:50:23.207182 19510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:23.207250 19512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:23.207226 19514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:50:23.208348 19505 server_base.cc:1034] running on GCE node
I20260810 14:50:23.209107 19505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:50:23.210451 19505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:50:23.211696 19505 hybrid_clock.cc:648] HybridClock initialized: now 1786373423211633 us; error 37 us; skew 500 ppm
I20260810 14:50:23.212030 19505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:50:23 dist-test-slave-vkjp krb5kdc[19324](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373423, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:50:23.215060 19505 init.cc:377] Logged in from keytab as kudu/127.24.215.188@KRBTEST.COM (short username kudu)
I20260810 14:50:23.216614 19505 webserver.cc:460] Webserver started at http://127.24.215.188:40551/ using document root <none> and password file <none>
I20260810 14:50:23.217329 19505 fs_manager.cc:359] Metadata directory not provided
I20260810 14:50:23.217463 19505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:50:23.217796 19505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:50:23.220841 19505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/data/instance:
uuid: "a6b1579b32f94e4b836bbf942630c51a"
format_stamp: "Formatted at 2026-08-10 14:50:23 on dist-test-slave-vkjp"
server_key: "91c9aab14a45150713fb247c257753af"
server_key_iv: "296d079fe28601dc67868b401e5afa9d"
server_key_version: "encryptionkey@0"
I20260810 14:50:23.221683 19505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/wal/instance:
uuid: "a6b1579b32f94e4b836bbf942630c51a"
format_stamp: "Formatted at 2026-08-10 14:50:23 on dist-test-slave-vkjp"
server_key: "91c9aab14a45150713fb247c257753af"
server_key_iv: "296d079fe28601dc67868b401e5afa9d"
server_key_version: "encryptionkey@0"
I20260810 14:50:23.226827 19505 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260810 14:50:23.230274 19526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:50:23.231689 19505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260810 14:50:23.231877 19505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/wal
uuid: "a6b1579b32f94e4b836bbf942630c51a"
format_stamp: "Formatted at 2026-08-10 14:50:23 on dist-test-slave-vkjp"
server_key: "91c9aab14a45150713fb247c257753af"
server_key_iv: "296d079fe28601dc67868b401e5afa9d"
server_key_version: "encryptionkey@0"
I20260810 14:50:23.232105 19505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:50:23.260649 19505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:50:23.264681 19505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:50:23.264966 19505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:50:23.284286 19505 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.188:38169
I20260810 14:50:23.284307 19586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.188:38169 every 8 connection(s)
I20260810 14:50:23.285648 19505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/data/info.pb
I20260810 14:50:23.289892 19587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:50:23.291661 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 19505
I20260810 14:50:23.291850 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786373051600012-25438-0/minicluster-data/master-2/wal/instance
I20260810 14:50:23.292304 25438 external_mini_cluster.cc:1428] Setting key bbe3809b606f3f2d39d10e560f5d7985
I20260810 14:50:23.299834 19587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:50:23 dist-test-slave-vkjp krb5kdc[19324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.188@KRBTEST.COM
Aug 10 14:50:23 dist-test-slave-vkjp krb5kdc[19324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373423, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.188@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:50:23.317562 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.294222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54694 (local address 127.24.215.188:38169)
0810 14:50:23.295119 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:23.295136 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:23.295193 (+ 57us) server_negotiation.cc:407] Connection header received
0810 14:50:23.296844 (+ 1651us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:23.296876 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:23.297418 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:23.297879 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:23.299288 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.301304 (+ 2016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.302856 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.304084 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.304266 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:23.309129 (+ 4863us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:23.309141 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:23.309154 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:23.309201 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:23.312650 (+ 3449us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.313371 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.313381 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.313391 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.313587 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.314018 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.314025 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.314029 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.314755 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:23.315056 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:23.315810 (+ 754us) server_negotiation.cc:299] Negotiation successful
0810 14:50:23.316151 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":349,"threads_started":1}
I20260810 14:50:23.319198 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.302985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:34261 (local address 127.0.0.1:53122)
0810 14:50:23.303946 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:23.304020 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:23.304193 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:23.304596 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:23.304601 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:23.305160 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:23.305403 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:23.305411 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.306887 (+ 1476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.306891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:23.308020 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:23.308032 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.309129 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.309138 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:23.309352 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:23.310045 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:23.310078 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:23.312743 (+ 2665us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:23.316469 (+ 3726us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:23.316479 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:23.316487 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:23.317031 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:23.317388 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:23.317393 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:23.317396 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:23.317584 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:23.318207 (+ 623us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:23.318212 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:23.318366 (+ 154us) client_negotiation.cc:769] Sending connection context
0810 14:50:23.318547 (+ 181us) client_negotiation.cc:240] Negotiation successful
0810 14:50:23.318757 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":562,"threads_started":1}
I20260810 14:50:23.319289 19593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.302948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53122 (local address 127.24.215.190:34261)
0810 14:50:23.303525 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:23.303535 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:23.304137 (+ 602us) server_negotiation.cc:407] Connection header received
0810 14:50:23.304315 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:23.304320 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:23.304421 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:23.304544 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:23.305561 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.306736 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.308177 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.308842 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.308977 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:23.312920 (+ 3943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:23.312925 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:23.312929 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:23.312967 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:23.316347 (+ 3380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.317153 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.317157 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.317159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.317279 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.317720 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.317727 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.317732 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.318077 (+ 345us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:23.318235 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:23.318716 (+ 481us) server_negotiation.cc:299] Negotiation successful
0810 14:50:23.318936 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":420,"thread_start_us":263,"threads_started":1}
I20260810 14:50:23.320705 19587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:50:23.324267 19414 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } attempt: 4
Aug 10 14:50:23 dist-test-slave-vkjp krb5kdc[19324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373423, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.188@KRBTEST.COM for kudu/127.24.215.189@KRBTEST.COM
Aug 10 14:50:23 dist-test-slave-vkjp krb5kdc[19324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
Aug 10 14:50:23 dist-test-slave-vkjp krb5kdc[19324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for kudu/127.24.215.188@KRBTEST.COM
I20260810 14:50:23.336529 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.321374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:41363 (local address 127.0.0.1:43130)
0810 14:50:23.321663 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:23.321683 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:23.321770 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:23.322079 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:23.322083 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:23.322463 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:23.322596 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:23.322604 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.323650 (+ 1046us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.323657 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:23.324754 (+ 1097us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:23.324769 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.325928 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.325933 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:23.326160 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:23.326829 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:23.326852 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:23.329328 (+ 2476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:23.333394 (+ 4066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:23.333402 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:23.333410 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:23.333889 (+ 479us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:23.334315 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:23.334320 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:23.334324 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:23.334494 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:23.335522 (+ 1028us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:23.335527 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:23.335751 (+ 224us) client_negotiation.cc:769] Sending connection context
0810 14:50:23.335927 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 14:50:23.336141 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171}
I20260810 14:50:23.336748 19502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.321562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43130 (local address 127.24.215.189:41363)
0810 14:50:23.321761 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:23.321767 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:23.321786 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:50:23.321860 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:23.321865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:23.321933 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:23.322042 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:23.322745 (+ 703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.323516 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.324957 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.325565 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.325680 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:23.329527 (+ 3847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:23.329534 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:23.329538 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:23.329577 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:23.333250 (+ 3673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.334011 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.334018 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.334022 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.334199 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.334843 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.334852 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.334855 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.335256 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:23.335453 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:23.336067 (+ 614us) server_negotiation.cc:299] Negotiation successful
0810 14:50:23.336303 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260810 14:50:23.337599 19587 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:50:23.339628 19593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.320839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53124 (local address 127.24.215.190:34261)
0810 14:50:23.320999 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:23.321004 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:23.321055 (+ 51us) server_negotiation.cc:407] Connection header received
0810 14:50:23.321234 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:23.321239 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:23.321342 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:23.321445 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:23.322818 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.323555 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.324904 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.325592 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.325703 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:23.330490 (+ 4787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:23.330497 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:23.330503 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:23.330560 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:23.334250 (+ 3690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.335191 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.335197 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.335201 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.335354 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.335807 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.335812 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.335816 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.336199 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:23.336452 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:23.339001 (+ 2549us) server_negotiation.cc:299] Negotiation successful
0810 14:50:23.339293 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260810 14:50:23.341775 19416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.324825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54696)
0810 14:50:23.325192 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:23.325211 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:23.325319 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:23.325676 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:23.325681 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:23.326174 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:23.326388 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:23.326406 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.327813 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.327819 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:23.329205 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:23.329218 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.330165 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.330173 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:23.330378 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:23.331000 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:23.331018 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:23.333624 (+ 2606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:23.338242 (+ 4618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:23.338249 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:23.338255 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:23.338936 (+ 681us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:23.339417 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:23.339422 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:23.339427 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:23.339598 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:23.340762 (+ 1164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:23.340767 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:23.340964 (+ 197us) client_negotiation.cc:769] Sending connection context
0810 14:50:23.341142 (+ 178us) client_negotiation.cc:240] Negotiation successful
0810 14:50:23.341429 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259}
I20260810 14:50:23.341845 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.325014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54696 (local address 127.24.215.188:38169)
0810 14:50:23.325186 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:23.325191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:23.325283 (+ 92us) server_negotiation.cc:407] Connection header received
0810 14:50:23.325435 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:23.325440 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:23.325523 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:23.325626 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:23.326537 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.327659 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.329357 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.329976 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.330126 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:23.333832 (+ 3706us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:23.333839 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:23.333844 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:23.333946 (+ 102us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:23.338089 (+ 4143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.339034 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.339041 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.339044 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.339233 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.339770 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.339778 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.339783 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.340633 (+ 850us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:23.340790 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:23.341271 (+ 481us) server_negotiation.cc:299] Negotiation successful
0810 14:50:23.341501 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":59,"spinlock_wait_cycles":27008}
Aug 10 14:50:23 dist-test-slave-vkjp krb5kdc[19324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373423, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.188@KRBTEST.COM for kudu/127.24.215.188@KRBTEST.COM
I20260810 14:50:23.348474 19414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10: Bootstrap starting.
Aug 10 14:50:23 dist-test-slave-vkjp krb5kdc[19324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373421, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.189@KRBTEST.COM
I20260810 14:50:23.352417 19414 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10: Neither blocks nor log segments found. Creating new log.
I20260810 14:50:23.353920 19414 log.cc:824] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10: Log is configured to *not* fsync() on all Append() calls
I20260810 14:50:23.354728 19594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.338387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54710)
0810 14:50:23.338716 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:23.338734 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:23.338841 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:23.339712 (+ 871us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:23.339716 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:23.340131 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:23.340252 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:23.340260 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.342461 (+ 2201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.342466 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:23.343600 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:23.343614 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.344376 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.344381 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:23.344654 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:23.345210 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:23.345235 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:23.347876 (+ 2641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:23.351753 (+ 3877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:23.351765 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:23.351769 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:23.352167 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:23.352647 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:23.352652 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:23.352655 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:23.352815 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:23.353516 (+ 701us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:23.353523 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:23.353725 (+ 202us) client_negotiation.cc:769] Sending connection context
0810 14:50:23.353901 (+ 176us) client_negotiation.cc:240] Negotiation successful
0810 14:50:23.354282 (+ 381us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":23}
I20260810 14:50:23.355137 19595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.338598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54710 (local address 127.24.215.188:38169)
0810 14:50:23.339488 (+ 890us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:23.339492 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:23.339506 (+ 14us) server_negotiation.cc:407] Connection header received
0810 14:50:23.339539 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:23.339543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:23.339588 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:23.339696 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:23.340412 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.341684 (+ 1272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.343759 (+ 2075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.344230 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.344370 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:23.348128 (+ 3758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:23.348133 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:23.348137 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:23.348192 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:23.351610 (+ 3418us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.352329 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.352332 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.352335 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.352460 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.352954 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.352960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.352966 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.353362 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:23.353516 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:23.354058 (+ 542us) server_negotiation.cc:299] Negotiation successful
0810 14:50:23.354456 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":211,"server-negotiator.queue_time_us":766,"thread_start_us":591,"threads_started":1}
I20260810 14:50:23.359961 19502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.341806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43140 (local address 127.24.215.189:41363)
0810 14:50:23.341994 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:23.341999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:23.342017 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:50:23.342073 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:23.342077 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:23.342127 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:23.342202 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:23.343854 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.344906 (+ 1052us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.346766 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.347434 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.347576 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:23.352876 (+ 5300us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:23.352884 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:23.352892 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:23.352932 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:23.356541 (+ 3609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.357575 (+ 1034us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.357581 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.357585 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.357763 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.358298 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.358304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.358307 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.358673 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:23.358960 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:23.359434 (+ 474us) server_negotiation.cc:299] Negotiation successful
0810 14:50:23.359650 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 14:50:23.360812 19587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a: Bootstrap starting.
I20260810 14:50:23.364454 19587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a: Neither blocks nor log segments found. Creating new log.
I20260810 14:50:23.364533 19414 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10: No bootstrap required, opened a new log
I20260810 14:50:23.365561 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:50:23.365840 19587 log.cc:824] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a: Log is configured to *not* fsync() on all Append() calls
I20260810 14:50:23.369190 19587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a: No bootstrap required, opened a new log
I20260810 14:50:23.368963 19387 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "eee1732542c54b5498b72d1298461d10"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53124
I20260810 14:50:23.372781 19414 raft_consensus.cc:348] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } }
I20260810 14:50:23.373124 19414 raft_consensus.cc:729] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eee1732542c54b5498b72d1298461d10, State: Initialized, Role: FOLLOWER
I20260810 14:50:23.373566 19414 consensus_queue.cc:260] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } }
I20260810 14:50:23.374541 19598 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } } }
I20260810 14:50:23.374931 19598 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:50:23.375708 19414 sys_catalog.cc:564] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:50:23.375878 19387 raft_consensus.cc:482] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 14:50:23.376077 19387 raft_consensus.cc:3037] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:50:23.380052 19387 raft_consensus.cc:504] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } }
I20260810 14:50:23.380221 19587 raft_consensus.cc:348] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } }
I20260810 14:50:23.380657 19587 raft_consensus.cc:729] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a6b1579b32f94e4b836bbf942630c51a, State: Initialized, Role: FOLLOWER
I20260810 14:50:23.381150 19587 consensus_queue.cc:260] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } }
I20260810 14:50:23.382237 19611 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } } }
I20260810 14:50:23.382846 19611 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:50:23.383551 19587 sys_catalog.cc:564] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:50:23.383523 19387 leader_election.cc:288] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [CANDIDATE]: Term 1 election: Requested vote from peers bfed2fbc273f4c47952d135432e1ee8a (127.24.215.189:41363), a6b1579b32f94e4b836bbf942630c51a (127.24.215.188:38169)
I20260810 14:50:23.383653 19558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eee1732542c54b5498b72d1298461d10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "a6b1579b32f94e4b836bbf942630c51a"
I20260810 14:50:23.384143 19558 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:50:23.390102 19558 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eee1732542c54b5498b72d1298461d10 in term 1.
I20260810 14:50:23.390734 19357 leader_election.cc:302] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a6b1579b32f94e4b836bbf942630c51a, eee1732542c54b5498b72d1298461d10; no voters:
I20260810 14:50:23.391330 19598 raft_consensus.cc:2781] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:50:23.418011 19598 raft_consensus.cc:686] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [term 1 LEADER]: Becoming Leader. State: Replica: eee1732542c54b5498b72d1298461d10, State: Running, Role: LEADER
I20260810 14:50:23.419755 19598 consensus_queue.cc:237] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } }
I20260810 14:50:23.391921 19337 master_runner.cc:186] Error getting master registration for 127.24.215.189:41363: OK, instance_id { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" instance_seqno: 1786373422726007 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:50:23.427278 19337 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260810 14:50:23.427753 19361 tablet.cc:2369] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:50:23.431571 19615 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [sys.catalog]: SysCatalogTable state changed. Reason: New leader eee1732542c54b5498b72d1298461d10. Latest consensus state: current_term: 1 leader_uuid: "eee1732542c54b5498b72d1298461d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } } }
I20260810 14:50:23.432520 19615 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [sys.catalog]: This master's current role is: LEADER
W20260810 14:50:23.434397 19612 catalog_manager.cc:1558] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:50:23.434528 19612 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:50:23.435480 19625 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:50:23.438220 19625 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260810 14:50:23.452133 19497 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } attempt: 5
I20260810 14:50:23.457973 19595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.415846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54712 (local address 127.24.215.188:38169)
0810 14:50:23.416075 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:23.416083 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:23.422161 (+ 6078us) server_negotiation.cc:407] Connection header received
0810 14:50:23.422543 (+ 382us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:23.422553 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:23.422667 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:23.422766 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:23.431573 (+ 8807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.432682 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.438188 (+ 5506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.439176 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.439362 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:23.450848 (+ 11486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:23.450856 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:23.450861 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:23.450908 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:23.454601 (+ 3693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.455447 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.455452 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.455455 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.455596 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.456060 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.456065 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.456068 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.456492 (+ 424us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:23.456790 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:23.457396 (+ 606us) server_negotiation.cc:299] Negotiation successful
0810 14:50:23.457630 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
Aug 10 14:50:23 dist-test-slave-vkjp krb5kdc[19324](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373422, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.189@KRBTEST.COM for kudu/127.24.215.188@KRBTEST.COM
I20260810 14:50:23.466224 19558 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a [term 1 FOLLOWER]: Refusing update from remote peer eee1732542c54b5498b72d1298461d10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:50:23.472848 19500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.452966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:38169 (local address 127.0.0.1:54720)
0810 14:50:23.453291 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:23.453315 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:23.453437 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:23.454515 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:23.454524 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:23.455193 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:23.455472 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:23.455487 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.457227 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.457236 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:23.459221 (+ 1985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:23.459242 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.461369 (+ 2127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.461378 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:23.461717 (+ 339us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:23.462431 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:23.462456 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:23.465303 (+ 2847us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:23.469728 (+ 4425us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:23.469736 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:23.469741 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:23.470283 (+ 542us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:23.470720 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:23.470725 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:23.470729 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:23.470936 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:23.471796 (+ 860us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:23.471803 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:23.471962 (+ 159us) client_negotiation.cc:769] Sending connection context
0810 14:50:23.472159 (+ 197us) client_negotiation.cc:240] Negotiation successful
0810 14:50:23.472448 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169}
I20260810 14:50:23.473845 19591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.453022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54720 (local address 127.24.215.188:38169)
0810 14:50:23.453774 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:23.453783 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:23.453807 (+ 24us) server_negotiation.cc:407] Connection header received
0810 14:50:23.453874 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:23.453882 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:23.453974 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:23.454090 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:23.455689 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.456996 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.459821 (+ 2825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.460847 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.461000 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:23.465517 (+ 4517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:23.465525 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:23.465530 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:23.465568 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:23.469263 (+ 3695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.470434 (+ 1171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.470440 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.470444 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.470582 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.471164 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.471170 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.471173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.471546 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:23.471727 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:23.473257 (+ 1530us) server_negotiation.cc:299] Negotiation successful
0810 14:50:23.473498 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":540}
I20260810 14:50:23.483950 19502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.430890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43144 (local address 127.24.215.189:41363)
0810 14:50:23.431135 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:23.431141 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:23.441882 (+ 10741us) server_negotiation.cc:407] Connection header received
0810 14:50:23.442067 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:23.442075 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:23.442192 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:23.442308 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:23.444772 (+ 2464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.446274 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.460403 (+ 14129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.461129 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.461238 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:23.476623 (+ 15385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:23.476632 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:23.476638 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:23.476679 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:23.480447 (+ 3768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.481354 (+ 907us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.481360 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.481364 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.481529 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.482002 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.482010 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.482013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.482406 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:23.482648 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:23.483339 (+ 691us) server_negotiation.cc:299] Negotiation successful
0810 14:50:23.483604 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260810 14:50:23.483053 19598 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.014s
I20260810 14:50:23.495913 19497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a: Bootstrap starting.
I20260810 14:50:23.504611 19593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:23.416628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53126 (local address 127.24.215.190:34261)
0810 14:50:23.416875 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:23.416884 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:23.426665 (+ 9781us) server_negotiation.cc:407] Connection header received
0810 14:50:23.427692 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:23.427701 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:23.427810 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:23.427902 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:23.432882 (+ 4980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.434151 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.439360 (+ 5209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:23.440095 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:23.440199 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:23.447059 (+ 6860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:23.447069 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:23.447109 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:23.447169 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:23.463050 (+ 15881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.493353 (+ 30303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.493365 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.493372 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.493627 (+ 255us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:23.497376 (+ 3749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:23.497387 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:23.497393 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:23.498036 (+ 643us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:23.498471 (+ 435us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:23.503516 (+ 5045us) server_negotiation.cc:299] Negotiation successful
0810 14:50:23.503939 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260810 14:50:23.517059 19532 tablet.cc:2369] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260810 14:50:23.519627 19357 outbound_call.cc:322] RPC callback for RPC call kudu.consensus.ConsensusService.UpdateConsensus -> {remote=127.24.215.188:38169, user_credentials={real_user=slave}} blocked reactor thread for 52133.6us
I20260810 14:50:23.533713 19497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a: Neither blocks nor log segments found. Creating new log.
I20260810 14:50:23.535423 19497 log.cc:824] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a: Log is configured to *not* fsync() on all Append() calls
I20260810 14:50:23.539862 19611 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a [sys.catalog]: SysCatalogTable state changed. Reason: New leader eee1732542c54b5498b72d1298461d10. Latest consensus state: current_term: 1 leader_uuid: "eee1732542c54b5498b72d1298461d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } } }
I20260810 14:50:23.540216 19611 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:50:23.543759 19628 mvcc.cc:205] Tried to move back new op lower bound from 7316985542473224192 to 7316985542342246400. Current Snapshot: MvccSnapshot[applied={T|T < 7316985542473224192}]
I20260810 14:50:23.545671 19497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a: No bootstrap required, opened a new log
I20260810 14:50:23.557905 19497 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } }
I20260810 14:50:23.558350 19497 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfed2fbc273f4c47952d135432e1ee8a, State: Initialized, Role: FOLLOWER
I20260810 14:50:23.558871 19497 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } }
I20260810 14:50:23.565861 19643 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } } }
I20260810 14:50:23.566283 19643 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:50:23.568162 19497 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:50:23.568748 19467 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:50:23.571300 19468 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "eee1732542c54b5498b72d1298461d10" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "bfed2fbc273f4c47952d135432e1ee8a"
I20260810 14:50:23.574751 19467 raft_consensus.cc:1261] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a [term 1 FOLLOWER]: Refusing update from remote peer eee1732542c54b5498b72d1298461d10: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:50:23.579695 19631 consensus_queue.cc:1035] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:50:23.581686 19468 raft_consensus.cc:2454] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eee1732542c54b5498b72d1298461d10 in term 1.
I20260810 14:50:23.587955 19625 catalog_manager.cc:1347] Generated new cluster ID: eb35e2cf1d0846efbbaaae8bb2304a0d
I20260810 14:50:23.588071 19625 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:50:23.588330 19598 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eee1732542c54b5498b72d1298461d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } } }
I20260810 14:50:23.588662 19598 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:23.589924 19611 sys_catalog.cc:455] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eee1732542c54b5498b72d1298461d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } } }
I20260810 14:50:23.590281 19611 sys_catalog.cc:458] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:50:23.649952 19650 catalog_manager.cc:1259] Loaded cluster ID: eb35e2cf1d0846efbbaaae8bb2304a0d
I20260810 14:50:23.650147 19650 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a: loading cluster ID for follower catalog manager: success
W20260810 14:50:23.651507 19650 catalog_manager.cc:1576] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260810 14:50:23.652547 19650 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:50:23.645689 19505 master_runner.cc:186] Error getting master registration for 127.24.215.189:41363: OK, instance_id { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" instance_seqno: 1786373422726007 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:50:23.660279 19505 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 14:50:23.672740 19643 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a [sys.catalog]: SysCatalogTable state changed. Reason: New leader eee1732542c54b5498b72d1298461d10. Latest consensus state: current_term: 1 leader_uuid: "eee1732542c54b5498b72d1298461d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } } }
I20260810 14:50:23.673075 19643 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:50:23.672765 19662 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "eee1732542c54b5498b72d1298461d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } } }
I20260810 14:50:23.673240 19662 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:50:23.678576 19615 sys_catalog.cc:455] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "eee1732542c54b5498b72d1298461d10" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eee1732542c54b5498b72d1298461d10" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 34261 } } peers { permanent_uuid: "bfed2fbc273f4c47952d135432e1ee8a" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 41363 } } peers { permanent_uuid: "a6b1579b32f94e4b836bbf942630c51a" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 38169 } } }
I20260810 14:50:23.678874 19615 sys_catalog.cc:458] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10 [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:23.684507 19625 catalog_manager.cc:1370] Generated new certificate authority record
W20260810 14:50:23.684902 19664 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:50:23.685077 19664 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:50:23.686156 19625 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:50:23.709771 19625 catalog_manager.cc:5868] T 00000000000000000000000000000000 P eee1732542c54b5498b72d1298461d10: Generated new TSK 0
I20260810 14:50:23.711056 19625 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 14:50:23.880496 19593 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:23.869255 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53130 (local address 127.24.215.190:34261)
0810 14:50:23.869502 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:23.869510 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:23.876768 (+ 7258us) server_negotiation.cc:407] Connection header received
0810 14:50:23.878371 (+ 1603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:23.878381 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:23.878489 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:23.878585 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:23.880181 (+ 1596us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53130: BlockingRecv error: recv got EOF from 127.0.0.1:53130 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260810 14:50:23.889853 19591 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:23.872321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54736 (local address 127.24.215.188:38169)
0810 14:50:23.872520 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:23.872526 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:23.887639 (+ 15113us) server_negotiation.cc:407] Connection header received
0810 14:50:23.887972 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:23.887980 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:23.888086 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:23.888177 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:23.889569 (+ 1392us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54736: BlockingRecv error: recv got EOF from 127.0.0.1:54736 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 14:50:23.892573 19502 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:23.876878 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43156 (local address 127.24.215.189:41363)
0810 14:50:23.877104 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:23.877111 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:23.890556 (+ 13445us) server_negotiation.cc:407] Connection header received
0810 14:50:23.891066 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:23.891072 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:23.891177 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:23.891250 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:23.892322 (+ 1072us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43156: BlockingRecv error: recv got EOF from 127.0.0.1:43156 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260810 14:50:23.935854 19502 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:23.925234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43172 (local address 127.24.215.189:41363)
0810 14:50:23.925453 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:23.925459 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:23.933532 (+ 8073us) server_negotiation.cc:407] Connection header received
0810 14:50:23.934075 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:23.934082 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:23.934216 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:23.934309 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:23.935533 (+ 1224us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43172: BlockingRecv error: recv got EOF from 127.0.0.1:43172 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260810 14:50:23.978683 19502 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:23.968748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43178 (local address 127.24.215.189:41363)
0810 14:50:23.968937 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:23.968944 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:23.976613 (+ 7669us) server_negotiation.cc:407] Connection header received
0810 14:50:23.977134 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:23.977142 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:23.977242 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:23.977353 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:23.978430 (+ 1077us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43178: BlockingRecv error: recv got EOF from 127.0.0.1:43178 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260810 14:50:24.036141 19502 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:24.025941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43186 (local address 127.24.215.189:41363)
0810 14:50:24.026176 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:24.026183 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:24.033744 (+ 7561us) server_negotiation.cc:407] Connection header received
0810 14:50:24.034371 (+ 627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:24.034378 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:24.034506 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:24.034601 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:24.035853 (+ 1252us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43186: BlockingRecv error: recv got EOF from 127.0.0.1:43186 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
W20260810 14:50:24.086560 19502 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:24.076644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43192 (local address 127.24.215.189:41363)
0810 14:50:24.076937 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:24.076945 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:24.084413 (+ 7468us) server_negotiation.cc:407] Connection header received
0810 14:50:24.084959 (+ 546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:24.084966 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:24.085069 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:24.085150 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:24.086283 (+ 1133us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43192: BlockingRecv error: recv got EOF from 127.0.0.1:43192 (error 108)
Metrics: {"server-negotiator.queue_time_us":149}
W20260810 14:50:24.134967 19502 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:24.132478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43206 (local address 127.24.215.189:41363)
0810 14:50:24.132690 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:24.132697 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:24.133326 (+ 629us) server_negotiation.cc:407] Connection header received
0810 14:50:24.133541 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:24.133548 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:24.133668 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:24.133809 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:24.134704 (+ 895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43206: BlockingRecv error: recv got EOF from 127.0.0.1:43206 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260810 14:50:24.157409 19502 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:24.154911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43214 (local address 127.24.215.189:41363)
0810 14:50:24.155137 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:24.155144 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:24.155748 (+ 604us) server_negotiation.cc:407] Connection header received
0810 14:50:24.155997 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:24.156008 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:24.156130 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:24.156253 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:24.157103 (+ 850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43214: BlockingRecv error: recv got EOF from 127.0.0.1:43214 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260810 14:50:24.207001 19502 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:24.193635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43226 (local address 127.24.215.189:41363)
0810 14:50:24.193862 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:24.193868 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:24.204862 (+ 10994us) server_negotiation.cc:407] Connection header received
0810 14:50:24.205439 (+ 577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:24.205446 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:24.205557 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:24.205641 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:24.206732 (+ 1091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43226: BlockingRecv error: recv got EOF from 127.0.0.1:43226 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260810 14:50:24.277900 19502 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:24.275714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43228 (local address 127.24.215.189:41363)
0810 14:50:24.275910 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:24.275917 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:24.276411 (+ 494us) server_negotiation.cc:407] Connection header received
0810 14:50:24.276590 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:24.276596 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:24.276692 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:24.276846 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:24.277641 (+ 795us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43228: BlockingRecv error: recv got EOF from 127.0.0.1:43228 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260810 14:50:24.414737 19502 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:24.412820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43232 (local address 127.24.215.189:41363)
0810 14:50:24.413074 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:24.413082 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:24.413332 (+ 250us) server_negotiation.cc:407] Connection header received
0810 14:50:24.413593 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:24.413597 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:24.413669 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:24.413783 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:24.414552 (+ 769us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43232: BlockingRecv error: recv got EOF from 127.0.0.1:43232 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260810 14:50:24.656287 19650 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a: acquiring CA information for follower catalog manager: success
I20260810 14:50:24.658143 19650 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a6b1579b32f94e4b836bbf942630c51a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260810 14:50:24.676810 19502 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:24.674706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43242 (local address 127.24.215.189:41363)
0810 14:50:24.674932 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:24.674938 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:24.675244 (+ 306us) server_negotiation.cc:407] Connection header received
0810 14:50:24.675476 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:24.675488 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:24.675575 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:24.675719 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:24.676576 (+ 857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43242: BlockingRecv error: recv got EOF from 127.0.0.1:43242 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260810 14:50:24.686939 19664 catalog_manager.cc:1259] Loaded cluster ID: eb35e2cf1d0846efbbaaae8bb2304a0d
I20260810 14:50:24.687039 19664 catalog_manager.cc:1552] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a: loading cluster ID for follower catalog manager: success
I20260810 14:50:24.689527 19664 catalog_manager.cc:1574] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a: acquiring CA information for follower catalog manager: success
I20260810 14:50:24.691496 19664 catalog_manager.cc:1602] T 00000000000000000000000000000000 P bfed2fbc273f4c47952d135432e1ee8a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:50:25.203286 19739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:25.197187 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43256 (local address 127.24.215.189:41363)
0810 14:50:25.197819 (+ 632us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:25.197826 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:25.197845 (+ 19us) server_negotiation.cc:407] Connection header received
0810 14:50:25.198043 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:25.198047 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:25.198211 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:25.198351 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:50:25.198852 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:25.199596 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:25.200983 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:25.201408 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:25.201559 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:25.202108 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:50:25.202632 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:50:25.202812 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:25.202870 (+ 58us) server_negotiation.cc:299] Negotiation successful
0810 14:50:25.202956 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":473,"thread_start_us":346,"threads_started":1}
I20260810 14:50:25.219823 19746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:25.213885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54740 (local address 127.24.215.188:38169)
0810 14:50:25.214607 (+ 722us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:25.214612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:25.214629 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:50:25.214728 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:25.214731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:25.214888 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:25.215050 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:50:25.215495 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:25.216183 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:25.217465 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:25.217891 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:25.217995 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:25.218471 (+ 476us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:50:25.219022 (+ 551us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:50:25.219244 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:25.219371 (+ 127us) server_negotiation.cc:299] Negotiation successful
0810 14:50:25.219461 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":603,"thread_start_us":502,"threads_started":1}
I20260810 14:50:25.234346 19746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:25.226813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54754 (local address 127.24.215.188:38169)
0810 14:50:25.227005 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:25.227011 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:25.228465 (+ 1454us) server_negotiation.cc:407] Connection header received
0810 14:50:25.228778 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:25.228784 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:25.228874 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:25.228942 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:50:25.229741 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:25.230510 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:25.232369 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:25.232933 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:25.233055 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:25.233590 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:50:25.233716 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:50:25.233799 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:25.233957 (+ 158us) server_negotiation.cc:299] Negotiation successful
0810 14:50:25.234047 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260810 14:50:25.235054 19739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:25.227462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43260 (local address 127.24.215.189:41363)
0810 14:50:25.227675 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:25.227681 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:25.228680 (+ 999us) server_negotiation.cc:407] Connection header received
0810 14:50:25.228965 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:25.228973 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:25.229070 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:25.229161 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:50:25.229815 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:25.230856 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:25.232943 (+ 2087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:25.233589 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:25.233722 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:25.234171 (+ 449us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:50:25.234336 (+ 165us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:50:25.234388 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:25.234655 (+ 267us) server_negotiation.cc:299] Negotiation successful
0810 14:50:25.234737 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260810 14:50:25.235139 19754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:25.226606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53144 (local address 127.24.215.190:34261)
0810 14:50:25.227907 (+ 1301us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:25.227916 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:25.227939 (+ 23us) server_negotiation.cc:407] Connection header received
0810 14:50:25.228099 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:25.228108 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:25.228360 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:25.228626 (+ 266us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:50:25.229193 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:25.230344 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:25.232650 (+ 2306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:25.233267 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:25.233409 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:25.233989 (+ 580us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:50:25.234479 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:50:25.234605 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:25.234709 (+ 104us) server_negotiation.cc:299] Negotiation successful
0810 14:50:25.234791 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1133,"thread_start_us":986,"threads_started":1}
I20260810 14:50:25.240859 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 19337
I20260810 14:50:25.241860 19409 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:50:25.357977 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 19337
W20260810 14:50:25.385727 19527 connection.cc:535] server connection from 127.0.0.1:54696 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 14:50:25.386480 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 19421
I20260810 14:50:25.387671 19492 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:50:25.494793 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 19421
I20260810 14:50:25.521570 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 19505
I20260810 14:50:25.522692 19581 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:50:25.634987 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 19505
2026-08-10T14:50:25Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5457 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 10 14:50:25 dist-test-slave-vkjp krb5kdc[19762](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 10 14:50:25 dist-test-slave-vkjp krb5kdc[19762](info): set up 2 sockets
Aug 10 14:50:25 dist-test-slave-vkjp krb5kdc[19762](info): commencing operation
krb5kdc: starting...
W20260810 14:50:26.852286 25438 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.044s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/test-user.keytab.
Aug 10 14:50:27 dist-test-slave-vkjp krb5kdc[19762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-10T14:50:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-10T14:50:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.24.215.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
Entry for principal kudu/127.24.215.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab.
I20260810 14:50:27.173655 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:44293
--webserver_interface=127.24.215.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:40313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:44293,127.24.215.189:36705,127.24.215.188:35795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260810 14:50:27.516352 19778 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:50:27.516688 19778 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:50:27.516769 19778 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:50:27.516851 19778 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:50:27.524288 19778 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:50:27.524386 19778 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:50:27.524444 19778 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:50:27.524497 19778 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:50:27.524562 19778 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:50:27.534541 19778 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:40313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:44293,127.24.215.189:36705,127.24.215.188:35795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.190.keytab
--principal=kudu/127.24.215.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.190:44293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.24.215.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:50:27.535912 19778 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:50:27.537199 19778 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:50:27.548885 19783 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:27.548880 19784 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:27.548955 19786 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:50:27.549785 19778 server_base.cc:1034] running on GCE node
I20260810 14:50:27.550630 19778 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:50:27.552090 19778 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:50:27.553349 19778 hybrid_clock.cc:648] HybridClock initialized: now 1786373427553283 us; error 55 us; skew 500 ppm
I20260810 14:50:27.553707 19778 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:50:27 dist-test-slave-vkjp krb5kdc[19762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373427, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:50:27.561764 19778 init.cc:377] Logged in from keytab as kudu/127.24.215.190@KRBTEST.COM (short username kudu)
I20260810 14:50:27.563482 19778 webserver.cc:460] Webserver started at http://127.24.215.190:39153/ using document root <none> and password file <none>
I20260810 14:50:27.564225 19778 fs_manager.cc:359] Metadata directory not provided
I20260810 14:50:27.564363 19778 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:50:27.564704 19778 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:50:27.568005 19778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/data/instance:
uuid: "55279481ccba4a19957d3d9b30ecf96e"
format_stamp: "Formatted at 2026-08-10 14:50:27 on dist-test-slave-vkjp"
server_key: "46de83cf5f3316edad76110019f461eb"
server_key_iv: "d1b87659bae9fb95d9eb830326f1c043"
server_key_version: "encryptionkey@0"
I20260810 14:50:27.568849 19778 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/wal/instance:
uuid: "55279481ccba4a19957d3d9b30ecf96e"
format_stamp: "Formatted at 2026-08-10 14:50:27 on dist-test-slave-vkjp"
server_key: "46de83cf5f3316edad76110019f461eb"
server_key_iv: "d1b87659bae9fb95d9eb830326f1c043"
server_key_version: "encryptionkey@0"
I20260810 14:50:27.574234 19778 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260810 14:50:27.577913 19796 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:50:27.579391 19778 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:50:27.579592 19778 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/wal
uuid: "55279481ccba4a19957d3d9b30ecf96e"
format_stamp: "Formatted at 2026-08-10 14:50:27 on dist-test-slave-vkjp"
server_key: "46de83cf5f3316edad76110019f461eb"
server_key_iv: "d1b87659bae9fb95d9eb830326f1c043"
server_key_version: "encryptionkey@0"
I20260810 14:50:27.579820 19778 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:50:27.600535 19778 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:50:27.604691 19778 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:50:27.604986 19778 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:50:27.624506 19778 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.190:44293
I20260810 14:50:27.624513 19852 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.190:44293 every 8 connection(s)
I20260810 14:50:27.625972 19778 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/data/info.pb
I20260810 14:50:27.630466 19853 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:50:27.634353 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 19778
I20260810 14:50:27.634552 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-0/wal/instance
I20260810 14:50:27.635139 25438 external_mini_cluster.cc:1428] Setting key 6cf4a9e575193cc7875c3b2a33de4bc1
I20260810 14:50:27.639771 19853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:50:27 dist-test-slave-vkjp krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373427, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:50:27.664824 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:27.643959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:44293 (local address 127.0.0.1:41140)
0810 14:50:27.645272 (+ 1313us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:27.645313 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:27.647219 (+ 1906us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:27.648205 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:27.648210 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:27.648795 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:27.649135 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:27.649152 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:27.650758 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:27.650766 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:27.652747 (+ 1981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:27.652766 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:27.653691 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:27.653701 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:27.653934 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:27.654660 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:27.654699 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:27.657411 (+ 2712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:27.660586 (+ 3175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:27.660597 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:27.660608 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:27.661009 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:27.661432 (+ 423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:27.661439 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:27.661444 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:27.661603 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:27.662366 (+ 763us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:27.662373 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:27.662589 (+ 216us) client_negotiation.cc:769] Sending connection context
0810 14:50:27.662829 (+ 240us) client_negotiation.cc:240] Negotiation successful
0810 14:50:27.663481 (+ 652us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1054,"mutex_wait_us":188,"thread_start_us":810,"threads_started":1}
I20260810 14:50:27.664866 19856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:27.643959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41140 (local address 127.24.215.190:44293)
0810 14:50:27.644879 (+ 920us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:27.644896 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:27.645476 (+ 580us) server_negotiation.cc:407] Connection header received
0810 14:50:27.647620 (+ 2144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:27.647631 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:27.647949 (+ 318us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:27.648177 (+ 228us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:27.649368 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:27.650600 (+ 1232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:27.652887 (+ 2287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:27.653428 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:27.653569 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:27.657644 (+ 4075us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:27.657652 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:27.657664 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:27.657692 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:27.660441 (+ 2749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:27.661167 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:27.661173 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:27.661175 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:27.661317 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:27.661718 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:27.661729 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:27.661732 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:27.662217 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:27.662431 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:27.662969 (+ 538us) server_negotiation.cc:299] Negotiation successful
0810 14:50:27.663355 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":661,"thread_start_us":431,"threads_started":1}
WARNING: no policy specified for kudu/127.24.215.189@KRBTEST.COM; defaulting to no policy
I20260810 14:50:27.668427 19853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:50:27.669502 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:27.669079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36705 (local address 127.0.0.1:39518)
0810 14:50:27.669230 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:27.669338 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260810 14:50:27.669790 19799 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36705: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.189@KRBTEST.COM" created.
I20260810 14:50:27.671376 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:27.670927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36705 (local address 127.0.0.1:39528)
0810 14:50:27.671112 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:27.671241 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 14:50:27.671563 19853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.189:36705: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36705: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.189.keytab.
Entry for principal kudu/127.24.215.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.189.keytab.
I20260810 14:50:27.710148 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.189:36705
--webserver_interface=127.24.215.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.189.keytab
--principal=kudu/127.24.215.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:40313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:44293,127.24.215.189:36705,127.24.215.188:35795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:50:27.720983 19853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } attempt: 1
I20260810 14:50:27.722373 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:27.721904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36705 (local address 127.0.0.1:39536)
0810 14:50:27.722121 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:27.722239 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260810 14:50:27.723428 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:27.723074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36705 (local address 127.0.0.1:39548)
0810 14:50:27.723232 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:27.723303 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260810 14:50:27.723675 19853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.189:36705: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36705: connect: Connection refused (error 111)
I20260810 14:50:27.791927 19853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } attempt: 2
I20260810 14:50:27.793171 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:27.792692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36705 (local address 127.0.0.1:39560)
0810 14:50:27.792927 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:27.793033 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 14:50:27.794243 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:27.793859 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36705 (local address 127.0.0.1:39564)
0810 14:50:27.794022 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:27.794108 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260810 14:50:27.794430 19853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.189:36705: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36705: connect: Connection refused (error 111)
I20260810 14:50:27.885670 19853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } attempt: 3
I20260810 14:50:27.886794 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:27.886368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36705 (local address 127.0.0.1:39566)
0810 14:50:27.886569 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:27.886663 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260810 14:50:27.887892 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:27.887503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36705 (local address 127.0.0.1:39582)
0810 14:50:27.887660 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:27.887755 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 14:50:27.888069 19853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.189:36705: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36705: connect: Connection refused (error 111)
I20260810 14:50:28.031309 19853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } attempt: 4
I20260810 14:50:28.032567 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.032017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36705 (local address 127.0.0.1:39598)
0810 14:50:28.032333 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.032422 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260810 14:50:28.033620 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.033261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36705 (local address 127.0.0.1:39606)
0810 14:50:28.033419 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.033503 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36705: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 14:50:28.033806 19853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.189:36705: Network error: Client connection negotiation failed: client connection to 127.24.215.189:36705: connect: Connection refused (error 111)
W20260810 14:50:28.055557 19860 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:50:28.056003 19860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:50:28.056139 19860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:50:28.056281 19860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:50:28.063792 19860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:50:28.063918 19860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:50:28.064049 19860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:50:28.064141 19860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:50:28.064234 19860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:50:28.074324 19860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:40313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:44293,127.24.215.189:36705,127.24.215.188:35795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.189.keytab
--principal=kudu/127.24.215.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.189:36705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.24.215.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:50:28.075753 19860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:50:28.077163 19860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:50:28.089845 19869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:28.089886 19866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:28.089845 19867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:50:28.090735 19860 server_base.cc:1034] running on GCE node
I20260810 14:50:28.091694 19860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:50:28.093102 19860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:50:28.094336 19860 hybrid_clock.cc:648] HybridClock initialized: now 1786373428094272 us; error 37 us; skew 500 ppm
I20260810 14:50:28.094782 19860 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:50:28 dist-test-slave-vkjp krb5kdc[19762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:50:28.098061 19860 init.cc:377] Logged in from keytab as kudu/127.24.215.189@KRBTEST.COM (short username kudu)
I20260810 14:50:28.099490 19860 webserver.cc:460] Webserver started at http://127.24.215.189:42415/ using document root <none> and password file <none>
I20260810 14:50:28.100193 19860 fs_manager.cc:359] Metadata directory not provided
I20260810 14:50:28.100278 19860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:50:28.100548 19860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:50:28.103605 19860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/data/instance:
uuid: "5ef80732e6b74fa096d85bace1715363"
format_stamp: "Formatted at 2026-08-10 14:50:28 on dist-test-slave-vkjp"
server_key: "6492bc034413cc827bdeec9c7d1499d1"
server_key_iv: "e5ca11c33d3fd9330bf3e1be3a9927b2"
server_key_version: "encryptionkey@0"
I20260810 14:50:28.104359 19860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/wal/instance:
uuid: "5ef80732e6b74fa096d85bace1715363"
format_stamp: "Formatted at 2026-08-10 14:50:28 on dist-test-slave-vkjp"
server_key: "6492bc034413cc827bdeec9c7d1499d1"
server_key_iv: "e5ca11c33d3fd9330bf3e1be3a9927b2"
server_key_version: "encryptionkey@0"
I20260810 14:50:28.109499 19860 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260810 14:50:28.112972 19876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:50:28.114292 19860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260810 14:50:28.114485 19860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/wal
uuid: "5ef80732e6b74fa096d85bace1715363"
format_stamp: "Formatted at 2026-08-10 14:50:28 on dist-test-slave-vkjp"
server_key: "6492bc034413cc827bdeec9c7d1499d1"
server_key_iv: "e5ca11c33d3fd9330bf3e1be3a9927b2"
server_key_version: "encryptionkey@0"
I20260810 14:50:28.114707 19860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:50:28.130525 19860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:50:28.134608 19860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:50:28.134900 19860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:50:28.153661 19860 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.189:36705
I20260810 14:50:28.153667 19935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.189:36705 every 8 connection(s)
I20260810 14:50:28.155170 19860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/data/info.pb
I20260810 14:50:28.158923 19936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:50:28.162881 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 19860
I20260810 14:50:28.163128 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-1/wal/instance
I20260810 14:50:28.163597 25438 external_mini_cluster.cc:1428] Setting key 4eb896296e39e6a851f4c6b6573eb3fb
I20260810 14:50:28.165648 19936 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:50:28 dist-test-slave-vkjp krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.189@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:50:28.187541 19938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.168457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41146 (local address 127.24.215.190:44293)
0810 14:50:28.168997 (+ 540us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:28.169002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:28.169987 (+ 985us) server_negotiation.cc:407] Connection header received
0810 14:50:28.171992 (+ 2005us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:28.172000 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:28.172097 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:28.172226 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:28.173809 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.174573 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.176185 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.176610 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.176749 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.180901 (+ 4152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:28.180906 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:28.180910 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:28.180949 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:28.183933 (+ 2984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.184962 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.184966 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.184968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.185091 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.185638 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.185641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.185643 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.185938 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:28.186053 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:28.187030 (+ 977us) server_negotiation.cc:299] Negotiation successful
0810 14:50:28.187265 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":427,"thread_start_us":299,"threads_started":1}
I20260810 14:50:28.189113 19939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.169057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:44293 (local address 127.0.0.1:41146)
0810 14:50:28.169820 (+ 763us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.169867 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:28.171705 (+ 1838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:28.172474 (+ 769us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:28.172483 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:28.173025 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:28.173583 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:28.173600 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.174749 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.174753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:28.176031 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:28.176042 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.176846 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.176854 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:28.177097 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.177803 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:28.177827 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:28.180663 (+ 2836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:28.184079 (+ 3416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:28.184091 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:28.184112 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:28.184835 (+ 723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:28.185213 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:28.185221 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:28.185228 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:28.185509 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:28.186070 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:28.186081 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:28.186607 (+ 526us) client_negotiation.cc:769] Sending connection context
0810 14:50:28.186966 (+ 359us) client_negotiation.cc:240] Negotiation successful
0810 14:50:28.187586 (+ 620us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":313,"threads_started":1}
I20260810 14:50:28.190861 19936 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.24.215.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.24.215.188@KRBTEST.COM" created.
Aug 10 14:50:28 dist-test-slave-vkjp krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.189@KRBTEST.COM for kudu/127.24.215.189@KRBTEST.COM
I20260810 14:50:28.206797 19939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.191547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36705 (local address 127.0.0.1:39608)
0810 14:50:28.191714 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.191734 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:28.191836 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:28.193030 (+ 1194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:28.193035 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:28.193479 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:28.193610 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:28.193618 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.194972 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.194979 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:28.195929 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:28.195941 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.196797 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.196803 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:28.197011 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.197524 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:28.197540 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:28.199776 (+ 2236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:28.204233 (+ 4457us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:28.204238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:28.204241 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:28.204669 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:28.205124 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:28.205128 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:28.205131 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:28.205279 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:28.206063 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:28.206067 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:28.206184 (+ 117us) client_negotiation.cc:769] Sending connection context
0810 14:50:28.206299 (+ 115us) client_negotiation.cc:240] Negotiation successful
0810 14:50:28.206512 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58}
I20260810 14:50:28.207500 19941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.191911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39608 (local address 127.24.215.189:36705)
0810 14:50:28.192611 (+ 700us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:28.192617 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:28.192645 (+ 28us) server_negotiation.cc:407] Connection header received
0810 14:50:28.192693 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:28.192699 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:28.192872 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:28.193032 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:28.193788 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.194837 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.196074 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.196652 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.196790 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.200012 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:28.200030 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:28.200051 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:28.200094 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:28.204051 (+ 3957us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.204822 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.204833 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.204839 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.204985 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.205394 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.205402 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.205407 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.205921 (+ 514us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:28.206139 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:28.206506 (+ 367us) server_negotiation.cc:299] Negotiation successful
0810 14:50:28.206810 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":340,"threads_started":1}
I20260810 14:50:28.209838 19936 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:50:28.210960 19939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.210528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56852)
0810 14:50:28.210682 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.210798 (+ 116us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260810 14:50:28.211310 19879 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
I20260810 14:50:28.212992 19939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.212532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56854)
0810 14:50:28.212731 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.212842 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
W20260810 14:50:28.213248 19936 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:35795: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.24.215.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.188.keytab.
Entry for principal kudu/127.24.215.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.188.keytab.
I20260810 14:50:28.236357 25438 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
/tmp/dist-test-taskvAcnHb/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.188:35795
--webserver_interface=127.24.215.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.188.keytab
--principal=kudu/127.24.215.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.24.215.148:40313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.24.215.190:44293,127.24.215.189:36705,127.24.215.188:35795
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260810 14:50:28.262633 19936 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } attempt: 1
I20260810 14:50:28.264144 19939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.263462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56866)
0810 14:50:28.263738 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.263887 (+ 149us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260810 14:50:28.265318 19939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.264844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56880)
0810 14:50:28.265052 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.265157 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 14:50:28.265539 19936 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:35795: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
I20260810 14:50:28.333086 19853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } attempt: 5
I20260810 14:50:28.333788 19936 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } attempt: 2
I20260810 14:50:28.334959 19939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.334455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56890)
0810 14:50:28.334653 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.334757 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260810 14:50:28.336232 19939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.335790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56892)
0810 14:50:28.335973 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.336061 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
W20260810 14:50:28.336555 19936 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:35795: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Aug 10 14:50:28 dist-test-slave-vkjp krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373427, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for kudu/127.24.215.189@KRBTEST.COM
I20260810 14:50:28.347221 19941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.333982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39610 (local address 127.24.215.189:36705)
0810 14:50:28.334187 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:28.334192 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:28.334291 (+ 99us) server_negotiation.cc:407] Connection header received
0810 14:50:28.334480 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:28.334488 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:28.334596 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:28.334725 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:28.335795 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.336898 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.338102 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.338520 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.338656 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.341853 (+ 3197us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:28.341858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:28.341861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:28.341892 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:28.344601 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.345271 (+ 670us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.345278 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.345287 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.345403 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.345792 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.345797 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.345801 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.346088 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:28.346247 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:28.346636 (+ 389us) server_negotiation.cc:299] Negotiation successful
0810 14:50:28.346852 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260810 14:50:28.347229 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.333860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36705 (local address 127.0.0.1:39610)
0810 14:50:28.334173 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.334194 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:28.334339 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:28.334769 (+ 430us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:28.334776 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:28.335351 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:28.335610 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:28.335630 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.337054 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.337060 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:28.337939 (+ 879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:28.337951 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.338672 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.338678 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:28.338914 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.339581 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:28.339599 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:28.341663 (+ 2064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:28.344751 (+ 3088us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:28.344757 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:28.344761 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:28.345145 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:28.345526 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:28.345531 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:28.345535 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:28.345678 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:28.346211 (+ 533us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:28.346216 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:28.346372 (+ 156us) client_negotiation.cc:769] Sending connection context
0810 14:50:28.346525 (+ 153us) client_negotiation.cc:240] Negotiation successful
0810 14:50:28.346846 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187}
I20260810 14:50:28.348807 19853 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:50:28.349840 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.349432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56894)
0810 14:50:28.349642 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.349715 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260810 14:50:28.350754 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.350422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56898)
0810 14:50:28.350585 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.350650 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260810 14:50:28.351045 19853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:35795: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
I20260810 14:50:28.402350 19853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } attempt: 1
I20260810 14:50:28.403574 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.403115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56900)
0810 14:50:28.403312 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.403406 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260810 14:50:28.404608 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.404261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56914)
0810 14:50:28.404413 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.404497 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260810 14:50:28.404819 19853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:35795: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
I20260810 14:50:28.427793 19936 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } attempt: 3
I20260810 14:50:28.429064 19939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.428575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56922)
0810 14:50:28.428801 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.428902 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260810 14:50:28.430189 19939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.429762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56926)
0810 14:50:28.429995 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.430072 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
W20260810 14:50:28.430397 19936 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:35795: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
I20260810 14:50:28.473052 19853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } attempt: 2
I20260810 14:50:28.474313 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.473784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56940)
0810 14:50:28.474043 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.474146 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260810 14:50:28.475363 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.474964 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56948)
0810 14:50:28.475175 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.475245 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260810 14:50:28.475572 19853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:35795: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
I20260810 14:50:28.573639 19936 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } attempt: 4
I20260810 14:50:28.574891 19939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.574405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56952)
0810 14:50:28.574644 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.574734 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260810 14:50:28.575932 19939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.575576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56966)
0810 14:50:28.575719 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.575791 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260810 14:50:28.576140 19936 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:35795: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
I20260810 14:50:28.581772 19853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } attempt: 3
I20260810 14:50:28.583333 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.582753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56970)
0810 14:50:28.582974 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.583070 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260810 14:50:28.584398 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.584022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56984)
0810 14:50:28.584178 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.584269 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260810 14:50:28.584594 19853 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.24.215.188:35795: Network error: Client connection negotiation failed: client connection to 127.24.215.188:35795: connect: Connection refused (error 111)
W20260810 14:50:28.587030 19945 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260810 14:50:28.587472 19945 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260810 14:50:28.587594 19945 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260810 14:50:28.587750 19945 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260810 14:50:28.595664 19945 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260810 14:50:28.595806 19945 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260810 14:50:28.595914 19945 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260810 14:50:28.596017 19945 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260810 14:50:28.596132 19945 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260810 14:50:28.606693 19945 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.24.215.148:40313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.24.215.190:44293,127.24.215.189:36705,127.24.215.188:35795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/krb5kdc/kudu_127.24.215.188.keytab
--principal=kudu/127.24.215.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.24.215.188:35795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.24.215.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 10 Aug 2026 13:46:55 UTC on ccycloud.quasar-skomnt.root.comops.site
build id 4577964
ASAN enabled
I20260810 14:50:28.608326 19945 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260810 14:50:28.609760 19945 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260810 14:50:28.622032 19950 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:28.622123 19954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260810 14:50:28.622085 19952 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260810 14:50:28.623018 19945 server_base.cc:1034] running on GCE node
I20260810 14:50:28.623878 19945 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260810 14:50:28.625342 19945 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260810 14:50:28.626585 19945 hybrid_clock.cc:648] HybridClock initialized: now 1786373428626514 us; error 49 us; skew 500 ppm
I20260810 14:50:28.626987 19945 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 10 14:50:28 dist-test-slave-vkjp krb5kdc[19762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260810 14:50:28.630227 19945 init.cc:377] Logged in from keytab as kudu/127.24.215.188@KRBTEST.COM (short username kudu)
I20260810 14:50:28.631891 19945 webserver.cc:460] Webserver started at http://127.24.215.188:35241/ using document root <none> and password file <none>
I20260810 14:50:28.632886 19945 fs_manager.cc:359] Metadata directory not provided
I20260810 14:50:28.633083 19945 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260810 14:50:28.633555 19945 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260810 14:50:28.637121 19945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/data/instance:
uuid: "efefba068836488a9af9ad9ec2a68928"
format_stamp: "Formatted at 2026-08-10 14:50:28 on dist-test-slave-vkjp"
server_key: "4874c603c285e3897032d65003d7fcdc"
server_key_iv: "d35fcbe6a5034dbe6f392e7a976caaf4"
server_key_version: "encryptionkey@0"
I20260810 14:50:28.637992 19945 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/wal/instance:
uuid: "efefba068836488a9af9ad9ec2a68928"
format_stamp: "Formatted at 2026-08-10 14:50:28 on dist-test-slave-vkjp"
server_key: "4874c603c285e3897032d65003d7fcdc"
server_key_iv: "d35fcbe6a5034dbe6f392e7a976caaf4"
server_key_version: "encryptionkey@0"
I20260810 14:50:28.643304 19945 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260810 14:50:28.646818 19961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260810 14:50:28.648211 19945 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260810 14:50:28.648406 19945 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/data,/tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/wal
uuid: "efefba068836488a9af9ad9ec2a68928"
format_stamp: "Formatted at 2026-08-10 14:50:28 on dist-test-slave-vkjp"
server_key: "4874c603c285e3897032d65003d7fcdc"
server_key_iv: "d35fcbe6a5034dbe6f392e7a976caaf4"
server_key_version: "encryptionkey@0"
I20260810 14:50:28.648674 19945 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260810 14:50:28.677194 19945 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260810 14:50:28.681514 19945 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260810 14:50:28.681802 19945 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260810 14:50:28.701535 19945 rpc_server.cc:307] RPC server started. Bound to: 127.24.215.188:35795
I20260810 14:50:28.701538 20021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.24.215.188:35795 every 8 connection(s)
I20260810 14:50:28.702996 19945 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/data/info.pb
I20260810 14:50:28.706985 20022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260810 14:50:28.708526 25438 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu as pid 19945
I20260810 14:50:28.708722 25438 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskvAcnHb/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786373051600012-25438-0/minicluster-data/master-2/wal/instance
I20260810 14:50:28.709200 25438 external_mini_cluster.cc:1428] Setting key 625eec29e8afc9a35a18fc7a29fdd6f6
I20260810 14:50:28.717072 20022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:50:28 dist-test-slave-vkjp krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.188@KRBTEST.COM
Aug 10 14:50:28 dist-test-slave-vkjp krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.188@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:50:28.733716 20025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.711183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56990 (local address 127.24.215.188:35795)
0810 14:50:28.712055 (+ 872us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:28.712068 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:28.712112 (+ 44us) server_negotiation.cc:407] Connection header received
0810 14:50:28.713733 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:28.713759 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:28.714217 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:28.714584 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:28.716550 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.718023 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.719948 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.721171 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.721374 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.725591 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:28.725602 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:28.725617 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:28.725672 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:28.728881 (+ 3209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.729738 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.729748 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.729756 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.729933 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.730314 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.730322 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.730325 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.731109 (+ 784us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:28.731436 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:28.731836 (+ 400us) server_negotiation.cc:299] Negotiation successful
0810 14:50:28.732312 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":639,"thread_start_us":421,"threads_started":1}
I20260810 14:50:28.735893 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.720226 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41150 (local address 127.24.215.190:44293)
0810 14:50:28.720900 (+ 674us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:28.720906 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:28.721606 (+ 700us) server_negotiation.cc:407] Connection header received
0810 14:50:28.721688 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:28.721692 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:28.721755 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:28.721876 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:28.722997 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.723787 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.725186 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.725741 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.725877 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.729594 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:28.729599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:28.729602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:28.729658 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:28.733152 (+ 3494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.733841 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.733846 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.733849 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.734016 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.734402 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.734406 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.734409 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.734766 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:28.734919 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:28.735448 (+ 529us) server_negotiation.cc:299] Negotiation successful
0810 14:50:28.735623 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":346,"threads_started":1}
I20260810 14:50:28.736080 20027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.720248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.190:44293 (local address 127.0.0.1:41150)
0810 14:50:28.721290 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.721344 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:28.721494 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:28.721943 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:28.721950 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:28.722585 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:28.722835 (+ 250us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:28.722844 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.723942 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.723953 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:28.725035 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:28.725047 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.725937 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.725942 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:28.726098 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.726851 (+ 753us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:28.726868 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:28.729402 (+ 2534us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:28.733295 (+ 3893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:28.733304 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:28.733309 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:28.733715 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:28.734145 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:28.734148 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:28.734151 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:28.734283 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:28.734897 (+ 614us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:28.734902 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:28.735065 (+ 163us) client_negotiation.cc:769] Sending connection context
0810 14:50:28.735325 (+ 260us) client_negotiation.cc:240] Negotiation successful
0810 14:50:28.735643 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":753,"thread_start_us":575,"threads_started":1}
I20260810 14:50:28.737870 20022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } has no permanent_uuid. Determining permanent_uuid...
Aug 10 14:50:28 dist-test-slave-vkjp krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.188@KRBTEST.COM for kudu/127.24.215.189@KRBTEST.COM
Aug 10 14:50:28 dist-test-slave-vkjp krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.190@KRBTEST.COM
I20260810 14:50:28.752147 20027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.738533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.189:36705 (local address 127.0.0.1:39624)
0810 14:50:28.738691 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.738711 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:28.738805 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:28.739166 (+ 361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:28.739173 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:28.739528 (+ 355us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:28.739661 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:28.739668 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.740720 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.740730 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:28.742170 (+ 1440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:28.742186 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.743160 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.743166 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:28.743408 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.743943 (+ 535us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:28.743962 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:28.746216 (+ 2254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:28.749436 (+ 3220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:28.749442 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:28.749454 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:28.749962 (+ 508us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:28.750328 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:28.750336 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:28.750340 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:28.750524 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:28.751203 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:28.751206 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:28.751389 (+ 183us) client_negotiation.cc:769] Sending connection context
0810 14:50:28.751575 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 14:50:28.751846 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260810 14:50:28.752355 19941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.738649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39624 (local address 127.24.215.189:36705)
0810 14:50:28.738826 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:28.738833 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:28.738855 (+ 22us) server_negotiation.cc:407] Connection header received
0810 14:50:28.738914 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:28.738920 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:28.738993 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:28.739123 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:28.739813 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.740480 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.742320 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.742957 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.743125 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.746434 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:28.746443 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:28.746449 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:28.746501 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:28.749289 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.750081 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.750085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.750087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.750214 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.750648 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.750652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.750655 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.751023 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:28.751216 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:28.751704 (+ 488us) server_negotiation.cc:299] Negotiation successful
0810 14:50:28.751955 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260810 14:50:28.753479 20022 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } has no permanent_uuid. Determining permanent_uuid...
I20260810 14:50:28.753844 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.737250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41158 (local address 127.24.215.190:44293)
0810 14:50:28.737416 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:28.737420 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:28.737474 (+ 54us) server_negotiation.cc:407] Connection header received
0810 14:50:28.737677 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:28.737681 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:28.737767 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:28.737877 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:28.739373 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.740078 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.741787 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.742413 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.742506 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.747241 (+ 4735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:28.747248 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:28.747254 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:28.747297 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:28.750491 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.751377 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.751382 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.751385 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.751540 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.752018 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.752023 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.752026 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.752385 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:28.752698 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:28.753270 (+ 572us) server_negotiation.cc:299] Negotiation successful
0810 14:50:28.753516 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 10 14:50:28 dist-test-slave-vkjp krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.188@KRBTEST.COM for kudu/127.24.215.188@KRBTEST.COM
I20260810 14:50:28.761802 19853 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } attempt: 4
Aug 10 14:50:28 dist-test-slave-vkjp krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373427, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.24.215.189@KRBTEST.COM
I20260810 14:50:28.769057 20027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.754189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:56998)
0810 14:50:28.754389 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.754409 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:28.754516 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:28.754920 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:28.754924 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:28.755356 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:28.755538 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:28.755551 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.756505 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.756513 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:28.757366 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:28.757376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.758309 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.758313 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:28.758493 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.759138 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:28.759160 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:28.761232 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:28.766076 (+ 4844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:28.766083 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:28.766090 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:28.766631 (+ 541us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:28.767062 (+ 431us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:28.767066 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:28.767072 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:28.767250 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:28.767925 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:28.767930 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:28.768119 (+ 189us) client_negotiation.cc:769] Sending connection context
0810 14:50:28.768283 (+ 164us) client_negotiation.cc:240] Negotiation successful
0810 14:50:28.768633 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63}
I20260810 14:50:28.769271 20025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.754443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56998 (local address 127.24.215.188:35795)
0810 14:50:28.754628 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:28.754635 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:28.754656 (+ 21us) server_negotiation.cc:407] Connection header received
0810 14:50:28.754705 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:28.754710 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:28.754785 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:28.754905 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:28.755699 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.756371 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.757533 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.758153 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.758277 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.761433 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:28.761441 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:28.761448 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:28.761489 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:28.765864 (+ 4375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.766770 (+ 906us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.766776 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.766779 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.766926 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.767395 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.767402 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.767407 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.767797 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:28.767961 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:28.768443 (+ 482us) server_negotiation.cc:299] Negotiation successful
0810 14:50:28.768827 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":101,"server-negotiator.queue_time_us":63}
I20260810 14:50:28.772161 19941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.755617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39638 (local address 127.24.215.189:36705)
0810 14:50:28.755795 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:28.755800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:28.755817 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:50:28.755902 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:28.755907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:28.755977 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:28.756096 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:28.757315 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.758383 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.759639 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.760087 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.760234 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.765563 (+ 5329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:28.765570 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:28.765573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:28.765608 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:28.768980 (+ 3372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.769972 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.769978 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.769982 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.770135 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.770606 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.770610 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.770614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.770994 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:28.771295 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:28.771560 (+ 265us) server_negotiation.cc:299] Negotiation successful
0810 14:50:28.771816 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
Aug 10 14:50:28 dist-test-slave-vkjp krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373427, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.190@KRBTEST.COM for kudu/127.24.215.188@KRBTEST.COM
I20260810 14:50:28.775879 20022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928: Bootstrap starting.
I20260810 14:50:28.777493 25438 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260810 14:50:28.780153 20022 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928: Neither blocks nor log segments found. Creating new log.
I20260810 14:50:28.781827 20022 log.cc:824] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928: Log is configured to *not* fsync() on all Append() calls
I20260810 14:50:28.784049 19855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.762447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:57012)
0810 14:50:28.762636 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.762661 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:28.762746 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:28.763567 (+ 821us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:28.763574 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:28.764138 (+ 564us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:28.764379 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:28.764393 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.766108 (+ 1715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.766114 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:28.767585 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:28.767600 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.768578 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.768584 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:28.768793 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.769421 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:28.769459 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:28.776648 (+ 7189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:28.780966 (+ 4318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:28.780972 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:28.780976 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:28.781424 (+ 448us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:28.781911 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:28.781916 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:28.781920 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:28.782123 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:28.782915 (+ 792us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:28.782923 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:28.783182 (+ 259us) client_negotiation.cc:769] Sending connection context
0810 14:50:28.783368 (+ 186us) client_negotiation.cc:240] Negotiation successful
0810 14:50:28.783709 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260810 14:50:28.785120 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.762562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57012 (local address 127.24.215.188:35795)
0810 14:50:28.763275 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:28.763279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:28.763297 (+ 18us) server_negotiation.cc:407] Connection header received
0810 14:50:28.763335 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:28.763339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:28.763393 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:28.763511 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:28.764574 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.765998 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.767739 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.768376 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.770946 (+ 2570us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.776915 (+ 5969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:28.776920 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:28.776923 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:28.776958 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:28.780798 (+ 3840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.781574 (+ 776us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.781601 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.781606 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.781784 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.782245 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.782253 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.782256 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.782702 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:28.784493 (+ 1791us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:28.784609 (+ 116us) server_negotiation.cc:299] Negotiation successful
0810 14:50:28.784767 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":595,"thread_start_us":335,"threads_started":1}
I20260810 14:50:28.780972 19827 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "55279481ccba4a19957d3d9b30ecf96e"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41158
I20260810 14:50:28.785642 20022 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928: No bootstrap required, opened a new log
I20260810 14:50:28.790926 19853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e: Bootstrap starting.
I20260810 14:50:28.793764 20022 raft_consensus.cc:348] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } }
I20260810 14:50:28.794126 19853 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e: Neither blocks nor log segments found. Creating new log.
I20260810 14:50:28.794236 20022 raft_consensus.cc:729] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efefba068836488a9af9ad9ec2a68928, State: Initialized, Role: FOLLOWER
I20260810 14:50:28.794734 20022 consensus_queue.cc:260] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } }
I20260810 14:50:28.795387 19853 log.cc:824] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e: Log is configured to *not* fsync() on all Append() calls
I20260810 14:50:28.795768 20030 sys_catalog.cc:455] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } } }
I20260810 14:50:28.796186 20030 sys_catalog.cc:458] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:50:28.796927 20022 sys_catalog.cc:564] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:50:28.798552 19853 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e: No bootstrap required, opened a new log
I20260810 14:50:28.805248 19853 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } }
I20260810 14:50:28.805734 19853 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55279481ccba4a19957d3d9b30ecf96e, State: Initialized, Role: FOLLOWER
I20260810 14:50:28.806278 19853 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } }
W20260810 14:50:28.808635 20043 catalog_manager.cc:1558] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260810 14:50:28.808784 20043 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260810 14:50:28.809252 20046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } } }
I20260810 14:50:28.809839 20046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:50:28.810488 19853 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:50:28.811264 19827 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260810 14:50:28.811479 19827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:50:28.814961 19827 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } }
I20260810 14:50:28.820317 19994 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "55279481ccba4a19957d3d9b30ecf96e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "efefba068836488a9af9ad9ec2a68928"
I20260810 14:50:28.820923 19994 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:50:28.806308 19945 master_runner.cc:186] Error getting master registration for 127.24.215.190:44293: OK, instance_id { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" instance_seqno: 1786373427618995 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:50:28.821332 19945 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 14:50:28.823619 19827 leader_election.cc:288] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [CANDIDATE]: Term 1 election: Requested vote from peers 5ef80732e6b74fa096d85bace1715363 (127.24.215.189:36705), efefba068836488a9af9ad9ec2a68928 (127.24.215.188:35795)
I20260810 14:50:28.824668 19994 raft_consensus.cc:2454] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55279481ccba4a19957d3d9b30ecf96e in term 1.
I20260810 14:50:28.825312 19799 leader_election.cc:302] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55279481ccba4a19957d3d9b30ecf96e, efefba068836488a9af9ad9ec2a68928; no voters:
I20260810 14:50:28.825911 20046 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [term 1 FOLLOWER]: Leader election won for term 1
I20260810 14:50:28.832890 20046 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [term 1 LEADER]: Becoming Leader. State: Replica: 55279481ccba4a19957d3d9b30ecf96e, State: Running, Role: LEADER
I20260810 14:50:28.833570 20046 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } }
I20260810 14:50:28.837730 20052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55279481ccba4a19957d3d9b30ecf96e. Latest consensus state: current_term: 1 leader_uuid: "55279481ccba4a19957d3d9b30ecf96e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } } }
I20260810 14:50:28.838193 20052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:28.847802 20056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260810 14:50:28.857170 20056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260810 14:50:28.857915 19803 tablet.cc:2369] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:50:28.867053 20026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.847833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41160 (local address 127.24.215.190:44293)
0810 14:50:28.848023 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:28.848031 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:28.848138 (+ 107us) server_negotiation.cc:407] Connection header received
0810 14:50:28.848437 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:28.848443 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:28.848541 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:28.848619 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:28.851015 (+ 2396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.852264 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.854462 (+ 2198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.855226 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.855342 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.859209 (+ 3867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:28.859217 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:28.859223 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:28.859291 (+ 68us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:28.863325 (+ 4034us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.864289 (+ 964us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.864297 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.864302 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.864480 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.864962 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.864967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.864971 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.865385 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:28.865677 (+ 292us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:28.866333 (+ 656us) server_negotiation.cc:299] Negotiation successful
0810 14:50:28.866601 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260810 14:50:28.875336 19936 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } attempt: 5
I20260810 14:50:28.882733 20072 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
Aug 10 14:50:28 dist-test-slave-vkjp krb5kdc[19762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786373428, etypes {rep=17 tkt=17 ses=17}, kudu/127.24.215.189@KRBTEST.COM for kudu/127.24.215.188@KRBTEST.COM
I20260810 14:50:28.893462 19939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.875911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.24.215.188:35795 (local address 127.0.0.1:57030)
0810 14:50:28.876327 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0810 14:50:28.876347 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0810 14:50:28.876450 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0810 14:50:28.877318 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0810 14:50:28.877323 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0810 14:50:28.877840 (+ 517us) client_negotiation.cc:189] Negotiated authn=SASL
0810 14:50:28.878042 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:28.878054 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.879439 (+ 1385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.879445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:28.880983 (+ 1538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0810 14:50:28.881013 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.882241 (+ 1228us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.882247 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0810 14:50:28.882506 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.883153 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0810 14:50:28.883177 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0810 14:50:28.885988 (+ 2811us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0810 14:50:28.890077 (+ 4089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:28.890085 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:28.890091 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:28.890671 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:28.891205 (+ 534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0810 14:50:28.891210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0810 14:50:28.891214 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0810 14:50:28.891384 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0810 14:50:28.892329 (+ 945us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0810 14:50:28.892335 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0810 14:50:28.892522 (+ 187us) client_negotiation.cc:769] Sending connection context
0810 14:50:28.892696 (+ 174us) client_negotiation.cc:240] Negotiation successful
0810 14:50:28.893041 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277}
I20260810 14:50:28.894539 20025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.876659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57030 (local address 127.24.215.188:35795)
0810 14:50:28.876868 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:28.876874 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:28.876891 (+ 17us) server_negotiation.cc:407] Connection header received
0810 14:50:28.876941 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:28.876945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:28.877029 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:28.877124 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:28.878208 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.879251 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.881206 (+ 1955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.881905 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.882021 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.886210 (+ 4189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:28.886218 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:28.886224 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:28.886263 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:28.889895 (+ 3632us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.890836 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.890844 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.890850 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.891023 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.891531 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.891537 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.891540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.892079 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:28.892247 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:28.893877 (+ 1630us) server_negotiation.cc:299] Negotiation successful
0810 14:50:28.894126 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67,"spinlock_wait_cycles":36736}
I20260810 14:50:28.901187 19936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363: Bootstrap starting.
I20260810 14:50:28.904158 20028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.860729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57016 (local address 127.24.215.188:35795)
0810 14:50:28.860940 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:28.860946 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:28.867992 (+ 7046us) server_negotiation.cc:407] Connection header received
0810 14:50:28.868257 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:28.868263 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:28.868368 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:28.868444 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:28.871201 (+ 2757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.872339 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.877827 (+ 5488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.878519 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.878613 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.891820 (+ 13207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:28.891828 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:28.891833 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:28.891867 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:28.895445 (+ 3578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.896389 (+ 944us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.896397 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.896402 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.896562 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.902217 (+ 5655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.902223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.902226 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.902629 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:28.902870 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:28.903562 (+ 692us) server_negotiation.cc:299] Negotiation successful
0810 14:50:28.903822 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260810 14:50:28.905675 19936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363: Neither blocks nor log segments found. Creating new log.
I20260810 14:50:28.887826 19778 master_runner.cc:186] Error getting master registration for 127.24.215.189:36705: OK, instance_id { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" instance_seqno: 1786373428148279 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260810 14:50:28.906845 19778 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260810 14:50:28.909451 19941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:28.848792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39654 (local address 127.24.215.189:36705)
0810 14:50:28.848987 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:28.848993 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:28.862039 (+ 13046us) server_negotiation.cc:407] Connection header received
0810 14:50:28.862418 (+ 379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:28.862424 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:28.862514 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:28.862590 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:28.874848 (+ 12258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.875898 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.881813 (+ 5915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:28.882534 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:28.882641 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:28.897174 (+ 14533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0810 14:50:28.897186 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0810 14:50:28.897191 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0810 14:50:28.897235 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0810 14:50:28.901237 (+ 4002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.906785 (+ 5548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.906791 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.906795 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.906979 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0810 14:50:28.907626 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0810 14:50:28.907632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0810 14:50:28.907636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0810 14:50:28.908009 (+ 373us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0810 14:50:28.908238 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:28.908850 (+ 612us) server_negotiation.cc:299] Negotiation successful
0810 14:50:28.909100 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260810 14:50:28.910696 19936 log.cc:824] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363: Log is configured to *not* fsync() on all Append() calls
I20260810 14:50:28.914144 19994 raft_consensus.cc:1261] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928 [term 1 FOLLOWER]: Refusing update from remote peer 55279481ccba4a19957d3d9b30ecf96e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:50:28.916209 20052 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [LEADER]: Connected to new peer: Peer: permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:50:28.922569 19936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363: No bootstrap required, opened a new log
W20260810 14:50:28.934044 19969 tablet.cc:2369] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260810 14:50:28.942636 19936 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } }
I20260810 14:50:28.943202 19936 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ef80732e6b74fa096d85bace1715363, State: Initialized, Role: FOLLOWER
I20260810 14:50:28.943770 19936 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } }
I20260810 14:50:28.945128 20076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } } }
I20260810 14:50:28.947026 20076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:50:28.954293 20030 sys_catalog.cc:455] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55279481ccba4a19957d3d9b30ecf96e. Latest consensus state: current_term: 1 leader_uuid: "55279481ccba4a19957d3d9b30ecf96e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } } }
I20260810 14:50:28.954651 20030 sys_catalog.cc:458] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:50:28.955753 19936 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363 [sys.catalog]: configured and running, proceeding with master startup.
I20260810 14:50:28.956017 19906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "55279481ccba4a19957d3d9b30ecf96e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "5ef80732e6b74fa096d85bace1715363"
I20260810 14:50:28.956565 19906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363 [term 0 FOLLOWER]: Advancing to term 1
I20260810 14:50:28.963289 19906 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55279481ccba4a19957d3d9b30ecf96e in term 1.
I20260810 14:50:28.963883 20052 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "55279481ccba4a19957d3d9b30ecf96e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } } }
I20260810 14:50:28.964185 20052 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:28.965207 19905 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363 [term 1 FOLLOWER]: Refusing update from remote peer 55279481ccba4a19957d3d9b30ecf96e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260810 14:50:28.967411 20046 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260810 14:50:28.975647 20056 catalog_manager.cc:1347] Generated new cluster ID: 97b0932d03204441a556a90c27a34bff
I20260810 14:50:28.975780 20056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260810 14:50:28.978466 20030 sys_catalog.cc:455] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "55279481ccba4a19957d3d9b30ecf96e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } } }
I20260810 14:50:28.978739 20030 sys_catalog.cc:458] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:50:29.001916 20076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55279481ccba4a19957d3d9b30ecf96e. Latest consensus state: current_term: 1 leader_uuid: "55279481ccba4a19957d3d9b30ecf96e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } } }
I20260810 14:50:29.001969 20093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "55279481ccba4a19957d3d9b30ecf96e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } } }
I20260810 14:50:29.002322 20076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:50:29.002324 20093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363 [sys.catalog]: This master's current role is: FOLLOWER
I20260810 14:50:29.014777 20046 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "55279481ccba4a19957d3d9b30ecf96e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55279481ccba4a19957d3d9b30ecf96e" member_type: VOTER last_known_addr { host: "127.24.215.190" port: 44293 } } peers { permanent_uuid: "5ef80732e6b74fa096d85bace1715363" member_type: VOTER last_known_addr { host: "127.24.215.189" port: 36705 } } peers { permanent_uuid: "efefba068836488a9af9ad9ec2a68928" member_type: VOTER last_known_addr { host: "127.24.215.188" port: 35795 } } }
I20260810 14:50:29.015131 20046 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e [sys.catalog]: This master's current role is: LEADER
I20260810 14:50:29.017201 20097 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260810 14:50:29.019580 20056 catalog_manager.cc:1370] Generated new certificate authority record
I20260810 14:50:29.021569 20056 catalog_manager.cc:1504] Loading token signing keys...
I20260810 14:50:29.037364 20056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55279481ccba4a19957d3d9b30ecf96e: Generated new TSK 0
I20260810 14:50:29.038583 20056 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260810 14:50:29.128345 20026 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:29.125525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41166 (local address 127.24.215.190:44293)
0810 14:50:29.125721 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:29.125727 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:29.126291 (+ 564us) server_negotiation.cc:407] Connection header received
0810 14:50:29.126605 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:29.126612 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:29.126723 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:29.126805 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:29.128079 (+ 1274us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41166: BlockingRecv error: recv got EOF from 127.0.0.1:41166 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260810 14:50:29.130807 19941 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:29.126735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39658 (local address 127.24.215.189:36705)
0810 14:50:29.126928 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:29.126935 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:29.129165 (+ 2230us) server_negotiation.cc:407] Connection header received
0810 14:50:29.129403 (+ 238us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:29.129409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:29.129507 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:29.129581 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:29.130579 (+ 998us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39658: BlockingRecv error: recv got EOF from 127.0.0.1:39658 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260810 14:50:29.134142 20025 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:29.132086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57036 (local address 127.24.215.188:35795)
0810 14:50:29.132278 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:29.132284 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:29.132491 (+ 207us) server_negotiation.cc:407] Connection header received
0810 14:50:29.132629 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:29.132635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:29.132758 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:29.132875 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:29.133888 (+ 1013us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57036: BlockingRecv error: recv got EOF from 127.0.0.1:57036 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260810 14:50:29.148190 19941 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:29.145496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39674 (local address 127.24.215.189:36705)
0810 14:50:29.145689 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:29.145695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:29.146465 (+ 770us) server_negotiation.cc:407] Connection header received
0810 14:50:29.146802 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:29.146809 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:29.146949 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:29.147064 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:29.147924 (+ 860us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39674: BlockingRecv error: recv got EOF from 127.0.0.1:39674 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260810 14:50:29.199955 19941 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:29.183853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39684 (local address 127.24.215.189:36705)
0810 14:50:29.184187 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:29.184197 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:29.198093 (+ 13896us) server_negotiation.cc:407] Connection header received
0810 14:50:29.198392 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:29.198398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:29.198515 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:29.198587 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:29.199779 (+ 1192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39684: BlockingRecv error: recv got EOF from 127.0.0.1:39684 (error 108)
Metrics: {"server-negotiator.queue_time_us":135}
W20260810 14:50:29.208364 19941 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:29.206090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39694 (local address 127.24.215.189:36705)
0810 14:50:29.206318 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:29.206328 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:29.206902 (+ 574us) server_negotiation.cc:407] Connection header received
0810 14:50:29.207147 (+ 245us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:29.207153 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:29.207252 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:29.207367 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:29.208125 (+ 758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39694: BlockingRecv error: recv got EOF from 127.0.0.1:39694 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260810 14:50:29.217788 19941 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:29.215483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39710 (local address 127.24.215.189:36705)
0810 14:50:29.215751 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:29.215760 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:29.216278 (+ 518us) server_negotiation.cc:407] Connection header received
0810 14:50:29.216511 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:29.216515 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:29.216584 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:29.216693 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:29.217536 (+ 843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39710: BlockingRecv error: recv got EOF from 127.0.0.1:39710 (error 108)
Metrics: {"server-negotiator.queue_time_us":115}
W20260810 14:50:29.231204 19941 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:29.229002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39724 (local address 127.24.215.189:36705)
0810 14:50:29.229230 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:29.229240 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:29.229784 (+ 544us) server_negotiation.cc:407] Connection header received
0810 14:50:29.230006 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:29.230016 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:29.230086 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:29.230197 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:29.230941 (+ 744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39724: BlockingRecv error: recv got EOF from 127.0.0.1:39724 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260810 14:50:29.252914 19941 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:29.250563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39740 (local address 127.24.215.189:36705)
0810 14:50:29.250814 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:29.250824 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:29.251388 (+ 564us) server_negotiation.cc:407] Connection header received
0810 14:50:29.251628 (+ 240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:29.251633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:29.251723 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:29.251849 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:29.252672 (+ 823us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39740: BlockingRecv error: recv got EOF from 127.0.0.1:39740 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
W20260810 14:50:29.295455 19941 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:29.293042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39746 (local address 127.24.215.189:36705)
0810 14:50:29.293277 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:29.293287 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:29.293836 (+ 549us) server_negotiation.cc:407] Connection header received
0810 14:50:29.294083 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:29.294087 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:29.294166 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:29.294279 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:29.295190 (+ 911us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39746: BlockingRecv error: recv got EOF from 127.0.0.1:39746 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260810 14:50:29.365254 19941 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:29.362960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39758 (local address 127.24.215.189:36705)
0810 14:50:29.363235 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:29.363244 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:29.363790 (+ 546us) server_negotiation.cc:407] Connection header received
0810 14:50:29.364053 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:29.364058 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:29.364135 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:29.364245 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:29.365023 (+ 778us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39758: BlockingRecv error: recv got EOF from 127.0.0.1:39758 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
W20260810 14:50:29.501590 19941 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:29.499386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39762 (local address 127.24.215.189:36705)
0810 14:50:29.499624 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:29.499633 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:29.500056 (+ 423us) server_negotiation.cc:407] Connection header received
0810 14:50:29.500310 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:29.500314 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:29.500391 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:29.500524 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:29.501357 (+ 833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39762: BlockingRecv error: recv got EOF from 127.0.0.1:39762 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260810 14:50:29.763451 19941 negotiation.cc:337] Failed RPC negotiation. Trace:
0810 14:50:29.761529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39776 (local address 127.24.215.189:36705)
0810 14:50:29.761769 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:29.761777 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:29.761979 (+ 202us) server_negotiation.cc:407] Connection header received
0810 14:50:29.762235 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:29.762240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:29.762311 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:29.762421 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0810 14:50:29.763236 (+ 815us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39776: BlockingRecv error: recv got EOF from 127.0.0.1:39776 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260810 14:50:29.811002 20043 catalog_manager.cc:1259] Loaded cluster ID: 97b0932d03204441a556a90c27a34bff
I20260810 14:50:29.811137 20043 catalog_manager.cc:1552] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928: loading cluster ID for follower catalog manager: success
I20260810 14:50:29.813673 20043 catalog_manager.cc:1574] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928: acquiring CA information for follower catalog manager: success
I20260810 14:50:29.815492 20043 catalog_manager.cc:1602] T 00000000000000000000000000000000 P efefba068836488a9af9ad9ec2a68928: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:50:30.021770 20097 catalog_manager.cc:1259] Loaded cluster ID: 97b0932d03204441a556a90c27a34bff
I20260810 14:50:30.021888 20097 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363: loading cluster ID for follower catalog manager: success
I20260810 14:50:30.024580 20097 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363: acquiring CA information for follower catalog manager: success
I20260810 14:50:30.026407 20097 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5ef80732e6b74fa096d85bace1715363: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260810 14:50:30.290215 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:30.281563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39784 (local address 127.24.215.189:36705)
0810 14:50:30.284212 (+ 2649us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:30.284219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:30.284241 (+ 22us) server_negotiation.cc:407] Connection header received
0810 14:50:30.284324 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:30.284331 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:30.284587 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:30.284800 (+ 213us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:50:30.285259 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:30.286199 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:30.287663 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:30.288177 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:30.288337 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:30.288927 (+ 590us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:50:30.289549 (+ 622us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:50:30.289724 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:30.289785 (+ 61us) server_negotiation.cc:299] Negotiation successful
0810 14:50:30.289872 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2490,"thread_start_us":2275,"threads_started":1}
I20260810 14:50:30.306913 20176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:30.296799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57044 (local address 127.24.215.188:35795)
0810 14:50:30.300698 (+ 3899us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:30.300702 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:30.300738 (+ 36us) server_negotiation.cc:407] Connection header received
0810 14:50:30.300846 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:30.300850 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:30.301011 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:30.301161 (+ 150us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:50:30.301741 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:30.302519 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:30.304598 (+ 2079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:30.305011 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:30.305106 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:30.305745 (+ 639us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:50:30.306291 (+ 546us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:50:30.306400 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:30.306566 (+ 166us) server_negotiation.cc:299] Negotiation successful
0810 14:50:30.306633 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3792,"thread_start_us":3673,"threads_started":1}
I20260810 14:50:30.324434 20183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:30.313649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41182 (local address 127.24.215.190:44293)
0810 14:50:30.314193 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:30.314200 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:30.314741 (+ 541us) server_negotiation.cc:407] Connection header received
0810 14:50:30.314903 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:30.314911 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:30.315216 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:30.315424 (+ 208us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:50:30.316328 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:30.317796 (+ 1468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:30.321169 (+ 3373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:30.321806 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:30.321936 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:30.322784 (+ 848us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:50:30.323275 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:50:30.323381 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:30.324011 (+ 630us) server_negotiation.cc:299] Negotiation successful
0810 14:50:30.324113 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":270,"threads_started":1}
I20260810 14:50:30.326277 20176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:30.318591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57048 (local address 127.24.215.188:35795)
0810 14:50:30.318817 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:30.318831 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:30.319484 (+ 653us) server_negotiation.cc:407] Connection header received
0810 14:50:30.319756 (+ 272us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:30.319761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:30.319857 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:30.319921 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:50:30.320729 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:30.321808 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:30.324107 (+ 2299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:30.324741 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:30.324833 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:30.325479 (+ 646us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:50:30.325605 (+ 126us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:50:30.325670 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:30.325921 (+ 251us) server_negotiation.cc:299] Negotiation successful
0810 14:50:30.326012 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260810 14:50:30.327536 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0810 14:50:30.314717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39792 (local address 127.24.215.189:36705)
0810 14:50:30.314920 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0810 14:50:30.314926 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0810 14:50:30.319144 (+ 4218us) server_negotiation.cc:407] Connection header received
0810 14:50:30.319437 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0810 14:50:30.319445 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0810 14:50:30.319552 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0810 14:50:30.319620 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0810 14:50:30.320543 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:30.321688 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:30.324147 (+ 2459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0810 14:50:30.324879 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0810 14:50:30.324992 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0810 14:50:30.325965 (+ 973us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0810 14:50:30.326109 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0810 14:50:30.326191 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0810 14:50:30.327125 (+ 934us) server_negotiation.cc:299] Negotiation successful
0810 14:50:30.327220 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260810 14:50:30.329447 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 19778
I20260810 14:50:30.330628 19848 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:50:30.476459 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 19778
W20260810 14:50:30.502851 19964 connection.cc:535] server connection from 127.0.0.1:57012 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260810 14:50:30.503352 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 19860
I20260810 14:50:30.504498 19931 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:50:30.613382 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 19860
I20260810 14:50:30.638109 25438 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu pid 19945
I20260810 14:50:30.639473 20016 generic_service.cc:196] Checking for leaks (request via RPC)
I20260810 14:50:30.748703 25438 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskvAcnHb/build/asan/bin/kudu with pid 19945
2026-08-10T14:50:30Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5112 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10570 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (379117 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST